Brazilian Journal of Probability and Statistics (2003), 17, pp. 91–105. c©Associação Brasileira de Estat́ıstica Expected posterior priors in factor analysis (original) (raw)

Bayesian computation of the intrinsic structure of factor analytic models

2009

The study of factor analytic models often has to address two important issues: (a) the determination of the "optimum" number of factors and (b) the derivation of a unique simple structure whose interpretation is easy and straightforward. The classical approach deals with these two tasks separately, and sometimes resorts to ad-hoc methods. This paper proposes a Bayesian approach to these two important issues, and adapts ideas from stochastic geometry and Bayesian finite mixture modelling to construct an ergodic Markov chain having the posterior distribution of the complete collection of parameters (including the number of factors) as its equilibrium distribution. The proposed method uses an Automatic Relevance Determination (ARD) prior as the device of achieving the desired simple structure. A Gibbs sampler updating scheme is then combined with the simulation of a continuous-time birth-and-death point process to produce a sampling scheme that efficiently explores the posterior distribution of interest. The MCMC sample path obtained from the simulated posterior then provides a flexible ingredient for most of the inferential tasks of interest. Illustrations on both artificial and real tasks are provided, while major difficulties and challenges are discussed, along with ideas for future improvements.

Stochastic determination of the intrinsic structure in Bayesian factor analysis

Statistical and Applied Mathematical Sciences Institute, …, 2004

The study of factor analytic models often has to address two important issues: (a) the determination of the "optimum" number of factors and (b) the derivation of a unique simple structure whose interpretation is easy and straightforward. The classical approach deals with these two tasks separately, and sometimes resorts to ad-hoc methods. This paper proposes a Bayesian approach to these two important issues, and adapts ideas from stochastic geometry and Bayesian finite mixture modelling to construct an ergodic Markov chain having the posterior distribution of the complete collection of parameters (including the number of factors) as its equilibrium distribution. The proposed method uses an Automatic Relevance Determination (ARD) prior as the device of achieving the desired simple structure. A Gibbs sampler updating scheme is then combined with the simulation of a continuous-time birth-and-death point process to produce a sampling scheme that efficiently explores the posterior distribution of interest. The MCMC sample path obtained from the simulated posterior then provides a flexible ingredient for most of the inferential tasks of interest. Illustrations on both artificial and real tasks are provided, while major difficulties and challenges are discussed, along with ideas for future improvements.

Heterogeneous factor analysis models: A bayesian approach

Psychometrika, 2002

Multilevel factor analysis models are widely used in the social sciences to account for heterogeneity in mean structures. In this paper we extend previous work on multilevel models to account for general forms of heterogeneity in confirmatory factor analysis models. We specify various models of mean and covariance heterogeneity in confirmatory factor analysis and develop Markov Chain Monte Carlo (MCMC) procedures to perform Bayesian inference, model checking, and model comparison. We test our methodology using synthetic data and data from a consumption emotion study. The results from synthetic data show that our Bayesian model perform well in recovering the true parameters and selecting the appropriate model. More importantly, the results clearly illustrate the consequences of ignoring heterogeneity. Specifically, we find that ignoring heterogeneity can lead to sign reversals of the factor covariances, inflation of factor variances and underappreciation of uncertainty in parameter estimates. The results from the emotion study show that subjects vary both in means and covariances. Thus traditional psychometric methods cannot fully capture the heterogeneity in our data.

On the identifiability of Bayesian factor analytic models

2020

A well known identifiability issue in factor analytic models is the invariance with respect to orthogonal transformations. This problem burdens the inference under a Bayesian setup, where Markov chain Monte Carlo (MCMC) methods are used to generate samples from the posterior distribution. We introduce a post-processing scheme in order to deal with rotation, sign and permutation invariance of the MCMC sample. The exact version of the contributed algorithm requires to solve 2q2^q2q assignment problems per (retained) MCMC iteration, where qqq denotes the number of factors of the fitted model. For large numbers of factors two approximate schemes based on simulated annealing are also discussed. We demonstrate that the proposed method leads to interpretable posterior distributions using synthetic and publicly available data from typical factor analytic models as well as mixtures of factor analyzers. An R package is available online at CRAN web-page.

Sequential Bayesian Inference for Factor Analysis

2022

We develop an efficient Bayesian sequential inference framework for factor analysis models observed via various data types, such as continuous, binary and ordinal data. In the continuous data case, where it is possible to marginalise over the latent factors, the proposed methodology tailors the Iterated Batch Importance Sampling (IBIS) of Chopin (2002) to handle such models and we incorporate Hamiltonian Markov Chain Monte Carlo. For binary and ordinal data, we develop an efficient IBIS scheme to handle the parameter and latent factors, combining with Laplace or Variational Bayes approximations. The methodology can be used in the context of sequential hypothesis testing via Bayes factors, which are known to have advantages over traditional null hypothesis testing. Moreover, the developed sequential framework offers multiple benefits even in non-sequential cases, by providing posterior distribution, model evidence and scoring rules (under the prequential framework) in one go, and by ...

Bayesian Analysis of Dynamic Factor Models Using Multivariate T Distribution

REVISTA BRASILEIRA DE BIOMETRIA

The multivariate t models are symmetric and have heavier tail than the normal distribution and produce robust inference procedures for applications. In this paper, the Bayesian estimation of a dynamic factor model is presented, where the factors follow a multivariate autoregressive model, using the multivariate t distribution. Since the multivariate t distribution is complex, it was represented in this work as a mix of the multivariate normal distribution and a square root of a chi-square distribution. This method allowed the complete dene of all the posterior distributions. The inference on the parameters was made taking a sample of the posterior distribution through a Gibbs Sampler. The convergence was veried through graphical analysis and the convergence diagnostics of Geweke (1992) and Raftery and Lewis (1992).

A note on variational Bayesian factor analysis

Neural networks : the official journal of the International Neural Network Society, 2009

Existing works on variational bayesian (VB) treatment for factor analysis (FA) model such as . Variational inference for Bayesian mixture of factor analysers. In Advances in neural information proceeding systems. Cambridge, MA: MIT Press; Nielsen, F. B. . Variational approach to factor analysis and related models. Master's thesis, The Institute of Informatics and Mathematical Modelling, Technical University of Denmark.] are found theoretically and empirically to suffer two problems: x penalize the model more heavily than BIC and y perform unsatisfactorily in low noise cases as redundant factors can not be effectively suppressed. A novel VB treatment is proposed in this paper to resolve the two problems and a simulation study is conducted to testify its improved performance over existing treatments.

Bayesian inference for graphical factor analysis models

Psychometrika, 2001

We de ne graphical factor analysis models as graphical Gaussian models with unobserved variables. We generalise factor analysis models by allowing the concentration matrix of the residuals to have non-zero o -diagonal elements. Allowing a structure of associations gives information about the correlation left unexplained by the unobserved variables, which can be used both in the con rmatory and exploratory context. We rst present a su cient condition for global identi ability of this class of models. We then propose a model selection procedure, based on the calculation of the posterior probabilities of the model. Given the analytical intractability of the latter, we construct a reversible jump Markov chain Monte Carlo method. A simulation study to illustrate these ideas is nally presented.