Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation – An Illustration and Tutorial (original) (raw)

Handbook of latent variable …, 2007

Structural equation models (SEMs) with latent variables are routinely used in social science research, and are of increasing importance in biomedical applications. Standard practice in implementing SEMs relies on frequentist methods. This chapter provides a simple and concise description of an alternative Bayesian approach. We provide a brief overview of the literature, describe a Bayesian specification of SEMs, and outline a Gibbs sampling strategy for model fitting. Bayesian inferences are illustrated through an industrialization and democratization case study from the literature. The Bayesian approach has some distinct advantages, due to the availability of samples from the joint posterior distribution of the model parameters and latent variables, that we highlight. These posterior samples provide important information not contained in the measurement and structural parameters. As is illustrated using the case study, this information can often provide valuable insight into structural relationships.

A Gentle Introduction to Bayesian Hierarchical Linear Regression Models

arXiv (Cornell University), 2021

Considering the flexibility and applicability of Bayesian modeling, in this work we revise the main characteristics of two hierarchical models in a regression setting. We study the full probabilistic structure of the models along with the full conditional distribution for each model parameter. Under our hierarchical extensions, we allow the mean of the second stage of the model to have a linear dependency on a set of covariates. The Gibbs sampling algorithms used to obtain samples when fitting the models are fully described and derived. In addition, we consider a case study in which the plant size is characterized as a function of nitrogen soil concentration and a grouping factor (farm).

8 Bayesian Structural Equation Modeling

Elsevier eBooks, 2007

Novel Bayesian methodology in multivariate problems

This dissertation involves developing novel Bayesian methodology for multivariate problems. In particular, it focuses on two contexts: shrinkage based variable selection in multivariate regression and simultaneous covariance estimation of multiple groups. Both these projects are centered around fully Bayesian inference schemes based on hierarchical modeling to capture context-specic features of the data and the development of computationally ecient estimation algorithm. Variable selection over a potentially large set of covariates in a linear model is quite popular. In the Bayesian context, common prior choices can lead to a posterior expectation of the regression coecients that is a sparse (or nearly sparse) vector with a few non-zero components, those covariates that are most important. The rst project extends the global-local shrinkage idea to a scenario where one wishes to model multiple response variables simultaneously. Here, we have developed a variable selection method for a K-outcome model (multivariate regression) that identies the most important covariates across all outcomes. The prior for all regression coecients is a mean zero normal with coecient-specic variance term that consists of a predictor-specic factor (shared local shrinkage parameter) and a model-specic factor (global shrinkage term) that diers in each model. The performance of our v modeling approach is evaluated through simulation studies and a data example. Covariance estimation for multiple groups is a key feature for drawing inference from a heterogeneous population. One should seek to share information about common features in the dependence structures across the various groups. In the second project, we introduce a novel approach for estimating the covariance matrices for multiple groups using a hierarchical latent factor model that shrinks the factor loadings across groups toward a global value. Using a spike and slab model on these loading coecients provides a level of sparsity in the global factor structure. Parameter estimation is accomplished through a Markov chain Monte Carlo scheme, and a model selection approach is used to determine the number of factors to use. Finally, a number of simulation studies and a data application are shown to demonstrate the performance of our methodology. vi

The Bayesian Approach to Multi-equation Econometric Model Estimation The Bayesian Approach to Multi-equation Econometric Model Estimation

The Bayesian approach to Statistical inference enables making probability statements about parameter (s) of interest in any research work. This paper presents a study of multi-equation Econometric model estimation from the flat or locally uniform prior Bayesian approach. Using a two-equation model containing one just-identified and one over-identified equation, we present a Bayesian analysis of multi-equation econometric model as well as a Monte Carlo study on it, using WinBUGS (windows version of the software: Bayesian analysis Using Gibbs Sampling). In studies involving the use of flat or locally uniform prior, it is the usual practice to specify the prior distribution in such a way that the variance is large. However, the definition of this "large" variance could vary among researchers. This research work considered three different variances (10, 100 and 1000) with particular focus on the Mean squared error of posterior estimates, looking for possible sensitivity to the...

A Tutorial on Bayesian Multi-Model Linear Regression with BAS and JASP

2020

Linear regression analyses commonly involve two consecutive stages of statistical inquiry. In the first stage, a single ‘best’ model is defined by a specific selection of relevant predictors; in the second stage, the regression coefficients of the winning model are used for prediction and for inference concerning the importance of the predictors. However, such second-stage inference ignores the model uncertainty from the first stage, resulting in overconfident parameter estimates that generalize poorly. These drawbacks can be overcome by model averaging, a technique that retains all models for inference, weighting each model’s contribution by its posterior probability. Although conceptually straightforward, model averaging is rarely used in applied research, possibly due to the lack of easily accessible software. To bridge the gap between theory and practice, we provide a tutorial on linear regression using Bayesian model averaging in JASP, based on the BAS package in R. Firstly, we...

Bayesian estimation in single-index models

Statistica Sinica

Single-index models offer a flexible semiparametric regression framework for high-dimensional predictors. Bayesian methods have never been proposed for such models. We develop a Bayesian approach incorporating some frequentist methods: B-splines approximate the link function, the prior on the index vector is Fishervon Mises, and regularization with generalized cross validation is adopted to avoid over-fitting the link function. A random walk Metropolis algorithm is used to sample from the posterior. Simulation results indicate that our procedure provides some improvement over the best frequentist method available. Two data examples are included.

Bayesian hierarchical modeling on covariance valued data

Stat

Analysis of structural and functional connectivity (FC) of human brains is of pivotal importance for diagnosis of cognitive ability. The Human Connectome Project (HCP) provides an excellent source of neural data across different regions of interest (ROIs) of the living human brain. Individual specific data were available from an existing analysis (Dai et al., 2017) in the form of time varying covariance matrices representing the brain activity as the subjects perform a specific task. As a preliminary objective of studying the heterogeneity of brain connectomics across the population, we develop a probabilistic model for a sample of covariance matrices using a scaled Wishart distribution. We stress here that our data units are available in the form of covariance matrices, and we use the Wishart distribution to create our likelihood function rather than its more common usage as a prior on covariance matrices. Based on empirical explorations suggesting the data matrices to have low effective rank, we further model the center of the Wishart distribution using an orthogonal factor model type decomposition. We encourage shrinkage towards a low rank structure through a novel shrinkage prior and discuss strategies to sample from the posterior distribution using a combination of Gibbs and slice sampling. We extend our modeling framework to a dynamic setting to detect change points. The efficacy of the approach is explored in various 1

A Tutorial on Bayesian Analysis of Count Data Using JAGS

Journal of Behavioral Data Science

In behavioral studies, the frequency of a particular behavior or event is often collected and the acquired data are referred to as count data. This tutorial introduces readers to Poisson regression models which is a more appropriate approach for such data. Meanwhile, count data with excessive zeros often occur in behavioral studies and models such as zero-inflated or hurdle models can be employed for handling zero-inflation in the count data. In this tutorial, we aim to cover the necessary fundamentals for these methods and equip readers with application tools of JAGS. Examples of the implementation of the models in JAGS from within R are provided for demonstration purposes.

Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation – An Illustration and Tutorial (original) (raw)

Related papers