Structural inference of hierarchies in networks (original) (raw)

Hierarchical structure and the prediction of missing links in networks

2008

Networks have in recent years emerged as an invaluable tool for describing and quantifying complex systems in many branches of science [1, 2, 3]. Recent studies suggest that networks often exhibit hierarchical organization, where vertices divide into groups that further subdivide into groups of groups, and so forth over multiple scales. In many cases these groups are found to correspond to known functional units, such as ecological niches in food webs, modules in biochemical networks (protein interaction networks, metabolic networks, or genetic regulatory networks), or communities in social networks [4, 5, 6, 7]. Here we present a general technique for inferring hierarchical structure from network data and demonstrate that the existence of hierarchy can simultaneously explain and quantitatively reproduce many commonly observed topological properties of networks, such as right-skewed degree distributions, high clustering coefficients, and short path lengths. We further show that knowledge of hierarchical structure can be used to predict missing connections in partially known networks with high accuracy, and for more general network structures than competing techniques [8]. Taken together, our results suggest that hierarchy is a central organizing principle of complex networks, capable of offering insight into many network phenomena.

Extracting hierarchies with overlapping structure from network data

Proceedings of the 2011 Winter Simulation Conference (WSC), 2011

Relationships between entities in many complex systems, such as the Internet and social networks, have a natural hierarchical organization. Understanding these inherent hierarchies is essential for creating models of these systems. Thus, there is a recent body of research concerning the extraction of hierarchies from networks. We propose a new method for modeling hierarchies through extracting the affiliations of the network. From these affiliations, we construct a lattice of the relationships between nodes. A principal advantage of our approach is that any overlapping community structures of the nodes within the network have a natural representation within the lattice. We then show an example of our method using a real data set.

Inferring the maximum likelihood hierarchy in social networks

2009

Abstract Individuals in social networks are often organized under some hierarchy such as a command structure. In many cases, when this structure is unknown, there is a need to discover hierarchical organization. In this paper, we propose a novel, simple, and flexible method based on maximum likelihood to infer social hierarchy from weighted social networks. We empirically evaluate our method against both simulated and real-world datasets and show that our approach accurately recovers the underlying, latent hierarchy.

Hierarchical Community Structure in Complex (Social) Networks

Acta Physica Polonica B Proceedings Supplement, 2014

The investigation of community structures in networks is a task of great importance in many disciplines, namely physics, sociology, biology and computer science, where systems are often represented as graphs. One of the challenges is to find local communities in a graph from a local viewpoint, in the absence of the access to global information, and to reproduce the subjective hierarchical vision for each vertex. In this paper we present the improvement of an information dynamics algorithm in which the label propagation of nodes is based on the Markovian flow of information in the network under cognitive-inspired constraints [1]. We introduced two more complex heuristics that allow to detect the hierarchical community structure of the networks from a source vertex or a community, adopting fixed values of model's parameters. Experimental results show that the proposed methods are efficient and well-behaved in both real-world and synthetic networks.

A new method for finding hierarchical subgroups from networks

Social Networks, 2010

We present a new method for decomposing a social network into an optimal number of hierarchical subgroups. With a perfect hierarchical subgroup defined as one in which every member is automorphically equivalent to each other, the method uses the REGGE algorithm to measure the similarities among nodes and applies the k-means method to group the nodes that have congruent profiles of dissimilarities with other nodes into various numbers of hierarchical subgroups. The best number of subgroups is determined by minimizing the intra-cluster variance of dissimilarity subject to the constraint that the improvement in going to more subgroups is better than a network whose n nodes are maximally dispersed in the n-dimensional space would achieve. We also describe a decomposability metric that assesses the deviation of a real network from the ideal one that contains only perfect hierarchical subgroups. Four well known network data sets are used to demonstrate the method and metric. These demonstrations indicate the utility of our approach and suggest how it can be used in a complementary way to Generalized Blockmodeling for hierarchical decomposition.

Extracting the hierarchical organization of complex systems

2007

Extracting understanding from the growing "sea" of biological and socioeconomic data is one of the most pressing scientific challenges facing us. Here, we introduce and validate an unsupervised method that is able to accurately extract the hierarchical organization of complex biological, social, and technological networks. We define an ensemble of hierarchically nested random graphs, which we use to validate the method. We then apply our method to real-world networks, including the air-transportation network, an electronic circuit, an email exchange network, and metabolic networks. We find that our method enables us to obtain an accurate multi-scale descriptions of a complex system. complex networks | hierarchical organization | multi-scale representation | cellular metabolism | transportation networks

HSN-PAM: Finding Hierarchical Probabilistic Groups from Large-Scale Networks

Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007), 2007

Real-world social networks are often hierarchical, reflecting the fact that some communities are composed of a few smaller, sub-communities. This paper describes a hierarchical Bayesian model based scheme, namely HSN-PAM (Hierarchical Social Network-Pachinko Allocation Model), for discovering probabilistic, hierarchical communities in social networks. This scheme is powered by a previously developed hierarchical Bayesian model. In this scheme, communities are classified into two categories: super-communities and regular-communities. Two different network encoding approaches are explored to evaluate this scheme on research collaborative networks, including CiteSeer and NanoSCI. The experimental results demonstrate that HSN-PAM is effective for discovering hierarchical community structures in large-scale social networks.

Hierarchical Characterization of Complex Networks

Journal of Statistical Physics, 2006

While the majority of approaches to the characterization of complex networks has relied on measurements considering only the immediate neighborhood of each network node, valuable information about the network topological properties can be obtained by considering further neighborhoods. The current work considers the concept of virtual hierarchies established around each node and the respectively defined hierarchical node degree and clustering coefficient (introduced in cond-mat/0408076), complemented by new hierarchical measurements, in order to obtain a powerful set of topological features of complex networks. The interpretation of such measurements is discussed, including an analytical study of the hierarchical node degree for random networks, and the potential of the suggested measurements for the characterization of complex networks is illustrated with respect to simulations of random, scale-free and regular network models as well as real data (airports, proteins and word associations). The enhanced characterization of the connectivity provided by the set of hierarchical measurements also allows the use of agglomerative clustering methods in order to obtain taxonomies of relationships between nodes in a network, a possibility which is also illustrated in the current article.

Finding Structure in Dynamic Networks

2009

Social creatures interact in diverse ways: forming groups, mating, sending emails, and sharing ideas. Some of the interactions are accidental while others are a consequence of the underlying explicit or implicit social structures [5, 6, 22, 28]. One of the most important questions in sociology is the identification of such structures, which are variously viewed as communities [8, 9, 11, 12, 24, 36], hierarchies [1, 10, 14, 23, 29], or “social profiles”[26].

Identifying Community Structures from Network Data via Maximum Likelihood Methods

The B.E. Journal of Theoretical Economics, 2000

In many economic situations it is of interest to know who interacts with whom. In international trade, for example, some theories predict that members of certaing groups will have a higher probability of trading with each other than with those in other groups. Based on a model of within and across group interactions, we describe, characterize, and implement, a new method for identifying trading or community structures from network data. The method is based on maximum likelihood estimation, a standard statistical tool. µ Copiµ c is at the Cowles Foundation at Yale University, at Cornell for helpful discussions; Pritha Dev for pointing out an error in a previous version, and Rik Pieters and Hans Baumgartner for making their data available.