Scale-free networks as entropy competition (original) (raw)

A statistical mechanics approach for scale-free networks and finite-scale networks

Chaos: An Interdisciplinary Journal of Nonlinear Science, 2007

We present a statistical mechanics approach for the description of complex networks. We first define an energy and an entropy associated to a degree distribution which have a geometrical interpretation. Next we evaluate the distribution which extremize the free energy of the network. We find two important limiting cases: a scale-free degree distribution and a finite-scale degree distribution. The size of the space of allowed simple networks given these distribution is evaluated in the large network limit. Results are compared with simulations of algorithms generating these networks.

Degree distribution of complex networks from statistical mechanics principles

2006

In this paper we describe the emergence of scale-free degree distributions from statistical mechanics principles. We define an energy associated to a degree sequence as the logarithm of the number of indistinguishable simple networks it is possible to draw given the degree sequence. Keeping fixed the total number of nodes and links, we show that the energy of scale-free distribution is much higher than the energy associated to the degree sequence of regular random graphs. This results unable us to estimate the annealed average of the number of distinguishable simple graphs it is possible to draw given a scale-free distribution with structural cutoff. In particular we shaw that this number for large networks is strongly suppressed for power -law exponent γ → 2.

Entropy distribution and condensation in random networks with a given degree distribution

Physical review. E, Statistical, nonlinear, and soft matter physics, 2014

The entropy of network ensembles characterizes the amount of information encoded in the network structure and can be used to quantify network complexity and the relevance of given structural properties observed in real network datasets with respect to a random hypothesis. In many real networks the degrees of individual nodes are not fixed but change in time, while their statistical properties, such as the degree distribution, are preserved. Here we characterize the distribution of entropy of random networks with given degree sequences, where each degree sequence is drawn randomly from a given degree distribution. We show that the leading term of the entropy of scale-free network ensembles depends only on the network size and average degree and that entropy is self-averaging, meaning that its relative variance vanishes in the thermodynamic limit. We also characterize large fluctuations of entropy that are fully determined by the average degree in the network. Finally, above a certain...

Shannon entropy and degree correlations in complex networks

Proceedings of the 10th …, 2010

A wide range of empirical networks -whether biological, technological, information-related or linguistic -generically exhibit important degree-degree anticorrelations (i.e., they are disassortative), the only exceptions usually being social ones, which tend to be positively correlated (assortative). With a view to understanding where this universality originates, we obtain the Shannon entropy of a network and find that the partition of maximum entropy does not in general correspond to uncorrelated networks but, in the case of heterogeneous (scale-free) degree distributions, to a certain disassortativity. This approach not only gives a parsimonious explanation to a long-standing question, but also provides a neutral model against which to compare experimental data, and thus determine whether there are specific correlating mechanisms at work among the forces behind the evolution of a given real-world network.

A thermodynamic view of networks

Comptes Rendus Biologies, 2006

Networks can be described by the frequency distribution of the number of links associated with each node (the degree of the node). Of particular interest are the power law distributions, which give rise to the so-called scale-free networks, and the distributions of the form of the simplified canonical law (SCL) introduced by Mandelbrot, which give what we shall call the Mandelbrot networks. Many dynamical methods have been obtained for the construction of scale-free networks, but no dynamical construction of Mandelbrot networks has been demonstrated. Here we develop a systematic technique to obtain networks with any given distribution of the degrees of the nodes. This is done using a thermodynamic approach in which we maximise the entropy associated with degree distribution of the nodes of the network subject to certain constraints. These constraints can be chosen systematically to produce the desired network architecture. For large networks we therefore replace a dynamical approach to the stationary state by a thermodynamical viewpoint. We use the method to generate scale-free and Mandelbrot networks with arbitrarily chosen parameters. We emphasise that this approach opens the possibility of insights into a thermodynamics of networks by suggesting thermodynamic relations between macroscopic variables for networks. To cite this article: D.

Entropy rate of non-equilibrium growing networks

2011

New entropy measures have been recently introduced for the quantification of the complexity of networks. Most of these entropy measures apply to static networks or to dynamical processes defined on static complex networks. In this paper we define the entropy rate of growing network models. This entropy rate quantifies how many labeled networks are typically generated by the growing network models. We analytically evaluate the difference between the entropy rate of growing tree network models and the entropy of tree networks that have the same asymptotic degree distribution. We find that the growing networks with linear preferential attachment generated by dynamical models are exponentially less than the static networks with the same degree distribution for a large variety of relevant growing network models. We study the entropy rate for growing network models showing structural phase transitions including models with non-linear preferential attachment. Finally, we bring numerical evidence that the entropy rate above and below the structural phase transitions follow a different scaling with the network size.

The Statistical Physics of Real-World Networks

Statistical physics is the natural framework to model complex networks. In the last twenty years, it has brought novel physical insights on a variety of emergent phenomena, such as self-organisation, scale invariance, mixed distributions and ensemble non-equivalence, which cannot be deduced from the behaviour of the individual constituents. At the same time, thanks to its deep connection with information theory, statistical physics and the principle of maximum entropy have led to the definition of null models reproducing some features of empirical networks, but otherwise as random as possible. We review here the statistical physics approach for complex networks and the null models for the various physical problems, focusing in particular on the analytic frameworks reproducing the local features of the network. We show how these models have been used to detect statistically significant and predictive structural patterns in real-world networks, as well as to reconstruct the network structure in case of incomplete information. We further survey the statistical physics frameworks that reproduce more complex, semi-local network features using Markov chain Monte Carlo sampling, and the models of generalised network structures such as multiplex networks, interacting networks and simplicial complexes. The science of networks has exploded in the Information Age thanks to the unprecedented production and storage of data on basically any human activity. Indeed, a network represents the simplest yet extremely effective way to model a large class of technological, social, economic and biological systems, as a set of entities (nodes) and of interactions (links) among them. These interactions do represent the fundamental degrees of freedom of the network, and can be of different types—undirected or directed, binary or valued (weighted)—depending on the nature of the system and the resolution used to describe it. Notably, most of the networks observed in the real world fall within the domain of complex systems, as they exhibit strong and complicated interaction patterns, and feature collective emergent phenomena that do not follow trivially from the behaviours of the individual entities [1]. For instance, many networks are scale-free [2–6], meaning that the number of links incident to a node (known as the node's degree) is fat-tailed distributed, sometimes following a power-law: most nodes have a few links, but a few nodes (the hubs) have many of them. The same happens for the distribution of the total weight of connections incident to a node (the node's strength) [7, 8]. Similarly, most real-world networks are organised into modules or feature a community structure [9, 10], and they possess high clustering—as nodes tend to create tightly linked groups, but are also small-world [11–13] as the distance (in terms of number of connections) amongst node pairs scales logarithmically with the system size. The observation of these universal features in complex networks has stimulated the development of a unifying mathematical language to model their structure and understand the dynamical processes taking place on them—such as the flow of traffic on the Internet or the spreading of either diseases or information in a population [14–16]. Two different approaches to network modelling can be pursued. The first one consists in identifying one or more microscopic mechanisms driving the formation of the network, and use them to define a dynamic model which can reproduce some of the emergent properties of real systems. The small-world model [11], the preferential attachment model [2], the fitness model [5], the relevance model [17] and many others follow this approach which is akin to kinetic theory. These models can handle only simple microscopic dynamics, and thus while providing good physical insights they need several refinements to give quantitatively accurate predictions. The other possible approach consists in identifying a set of characteristic static properties of real systems, and then building networks having the same properties but otherwise maximally random. This approach is thus akin to statistical mechanics and therefore is based on rigorous probabilistic arguments that can lead to accurate and reliable predictions. The mathematical framework is that of exponential random graphs (ERG), which has been first introduced in the social sciences and statistics [18–26] as a convenient formulation relying on numerical techniques such as Markov chain Monte Carlo algorithms. The interpretation of ERG in physical terms is due to Park and Newman [27], who showed how to derive them from the principle of maximum entropy and the statistical mechanics of Boltzmann and Gibbs. As formulated by Jaynes [28], the variational principle of maximum entropy states that the probability distribution best representing the current state of (knowledge on) a system is the one which maximises the Shannon entropy, subject in principle to any prior information on the system itself. This means making self-consistent inference assum

Entropy measures for networks: Toward an information theory of complex topologies

Physical Review E, 2009

The quantification of the complexity of networks is, today, a fundamental problem in the physics of complex systems. A possible roadmap to solve the problem is via extending key concepts of information theory to networks. In this paper we propose how to define the Shannon entropy of a network ensemble and how it relates to the Gibbs and von Neumann entropies of network ensembles. The quantities we introduce here will play a crucial role for the formulation of null models of networks through maximum-entropy arguments and will contribute to inference problems emerging in the field of complex networks. PACS numbers: 89.75.Hc, 89.75.Fb, 89.75.Da Complex networks are found to characterize the underlying structure of many biological, social and technological systems. Following ten years of active research in the field of complex networks, the state of the art includes, a deep understanding of their evolution [1], an unveiling of the rich interplay between network topology and dynamics [3] and a description of networks through structural characteristics . Nevertheless, we still lack the means to quantify, how complex is a complex network. In order to answer this question we need a new theory of information of complex networks. This new theory will contribute to solving many challenging inference problems in the field [4, 5, 6]. By providing an evaluation of the information encoded in complex networks, this will resolve one of the outstanding problems in the statistical mechanics of complex systems.

Revisiting “scale-free” networks

BioEssays, 2005

Recent observations of power-law distributions in the connectivity of complex networks came as a big surprise to researchers steeped in the tradition of random networks. Even more surprising was the discovery that power-law distributions also characterize many biological and social networks. Many attributed a deep significance to this fact, inferring a ''universal architecture'' of complex systems. Closer examination, however, challenges the assumptions that (1) such distributions are special and (2) they signify a common architecture, independent of the system's specifics. The real surprise, if any, is that power-law distributions are easy to generate, and by a variety of mechanisms. The architecture that results is not universal, but particular; it is determined by the actual constraints on the system in question.