Fisher information matrix and hyperbolic geometry (original) (raw)
Related papers
Fisher information distance: A geometrical reading
Discrete Applied Mathematics, 2014
This paper is a strongly geometrical approach to the Fisher distance, which is a measure of dissimilarity between two probability distribution functions. The Fisher distance, as well as other divergence measures, are also used in many applications to establish a proper data average. The main purpose is to widen the range of possible interpretations and relations of the Fisher distance and its associated geometry for the prospective applications. It focuses on statistical models of the normal probability distribution functions and takes advantage of the connection with the classical hyperbolic geometry to derive closed forms for the Fisher distance in several cases. Connections with the well-known Kullback-Leibler divergence measure are also devised.
Transformed statistical distance measures and the fisher information matrix
Linear Algebra and Its Applications, 2012
Most multivariate statistical techniques are based upon the concept of distance. The purpose of this paper is to introduce statistical distance measures, which are normalized Euclidean distance measures, where the covariances of observed correlated measurements x 1 , . . . , x n and entries of the Fisher information matrix (FIM) are used as weighting coefficients. The measurements are subject to random fluctuations of different magnitudes and have therefore different variabilities. A rotation of the coordinate system through a chosen angle while keeping the scatter of points given by the data fixed, is therefore considered. It is shown that when the FIM is positive definite, the appropriate statistical distance measure is a metric. In case of a singular FIM, the metric property depends on the rotation angle. The introduced statistical distance measures, are matrix related, and are based on m parameters unlike a statistical distance measure in quantum information, which is also related to the Fisher information and where the information about one parameter in a particular measurement procedure is considered. A transformed FIM of a stationary process as well as the Sylvester resultant matrix are used to ensure the relevance of the appropriate statistical distance measure. The approach used in this paper is such that matrix properties are crucial for ensuring the relevance of the introduced statistical distance measures.
Entropy
The Fisher–Rao distance is a measure of dissimilarity between probability distributions, which, under certain regularity conditions of the statistical model, is up to a scaling factor the unique Riemannian metric invariant under Markov morphisms. It is related to the Shannon entropy and has been used to enlarge the perspective of analysis in a wide variety of domains such as image processing, radar systems, and morphological classification. Here, we approach this metric considered in the statistical model of normal multivariate probability distributions, for which there is not an explicit expression in general, by gathering known results (closed forms for submanifolds and bounds) and derive expressions for the distance between distributions with the same covariance matrix and between distributions with mirrored covariance matrices. An application of the Fisher–Rao distance to the simplification of Gaussian mixtures using the hierarchical clustering algorithm is also presented.
Relations between Kullback-Leibler distance and Fisher information
2002
The Kullback-Leibler distance between two probability densities that are parametric perturbations of each other is related to the Fisher information. We generalize this relationship to the case when the perturbations may not be small and when the two densities are non-parametric.
A Unifying Framework for Some Directed Distances in Statistics
ArXiv, 2022
Density-based directed distances — particularly known as divergences — between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likelihood estimation method, and Pearson’s χ ́distance which e.g. is used for the celebrated chisquare goodness-of-fit test. Another line of statistical inference is built upon distribution-function-based divergences such as e.g. the prominent (weighted versions of) Cramer-von Mises test statistics respectively AndersonDarling test statistics which are frequently applied for goodness-of-fit investigations; some more recent methods deal with (other kinds of) cumulative paired divergences and closely related concepts. In this paper, we provide a general framework which covers in particular both the ab...
On bounds for the Fisher-Rao distance between multivariate normal distributions
2015
Information geometry is approached here by considering the statistical model of multivariate normal distributions as a Riemannian manifold with the natural metric provided by the Fisher information matrix. Explicit forms for the Fisher-Rao distance associated to this metric and for the geodesics of general distribution models are usually very hard to determine. In the case of general multivariate normal distributions lower and upper bounds have been derived. We approach here some of these bounds and introduce a new one discussing their tightness in specific cases.
Information Sciences, 1995
In this paper, a transformation of Csiszar's measures which generalizes the unified (r, s) measures defined by Sharma and Mittal and Taneja is presented. For these transformations, information matrices associated to a differential metric in the direction to the tangent space are obtained, as well as the amount of information resulting from parameter perturbation in the direction of coordinate axes. Finally, the asymptotic distribution of information matrices and the amount of information and its applications to test statistical hypotheses are obtained. < F(x), I 00~ 00j < Y(x), 00~ 00j 00k < ~/(x), where F is finitely integrable and E[H(X)] < M, with M independent of 0. (iii) The Fisher information matrix O0 i OOj i,j = 1 ..
Connections of generalized divergence measures with Fisher information matrix
Information Sciences, 1993
In this paper parametric measures of information for the multiparameter case are obtained using Csiszar's nonparametric measure of information as well as continuous and differentiable functions thereof. The same idea is applied to some of the unified b, s) divergence measures presented in Taneja. Relationships of the new parametric measures with Fisher's information matrix are derived and their connections with the Cramer-Rao inequality is established. This work generalizes the results of Kagan, Aggarwal, and Ferentinos and Papioannou.
On the Fisher Information Matrix for Multivariate Elliptically Contoured Distributions
IEEE Signal Processing Letters, 2000
The Slepian-Bangs formula provides a very convenient way to compute the Fisher information matrix (FIM) for Gaussian distributed data. The aim of this letter is to extend it to a larger family of distributions, namely elliptically contoured (EC) distributions. More precisely, we derive a closed-form expression of the FIM in this case. This new expression involves the usual term of the Gaussian FIM plus some corrective factors that depend only on the expectations of some functions of the so-called modular variate. Hence, for most distributions in the EC family, derivation of the FIM from its Gaussian counterpart involves slight additional derivations. We show that the new formula reduces to the Slepian-Bangs formula in the Gaussian case and we provide an illustrative example with Student distributions on how it can be used.
Directed Divergence as a Measure of Similarity
International Journal of Pure and Apllied Mathematics, 2014
The development of interrelationships between divergence measures and the known statistical constants provide the applications of information theory to the field of statistics. In the literature of information measures, there exist many divergence measures for discrete probability distributions whereas we need such divergence measures for continuous distributions to extend the scope of their applications. In the present communication, we have introduced divergence measures for continuous variate distributions and then proved that the divergence between the joint distribution density and the product of the marginal distribution densities is a function of the correlation coefficient which obviously implies that the divergence is also a measure of the similarity or of the dissimilarity.