Catalytic Priors: Using Synthetic Data to Specify Prior Distributions in Bayesian Analysis (original) (raw)
Related papers
Catalytic prior distributions with application to generalized linear models
Proceedings of the National Academy of Sciences, 2020
Significance We propose a strategy for building prior distributions that stabilize the estimation of complex “working models” when sample sizes are too small for standard statistical analysis. The stabilization is achieved by supplementing the observed data with a small amount of synthetic data generated from the predictive distribution of a simpler model. This class of prior distributions is easy to use and allows direct statistical interpretation.
On information, priors, econometrics, and economic modeling
Estudios Económicos, 1999
Resumen: Se busca reconciliar aquellos métodos inferenciales que a través de la maximización de una funcional producen distribuciones a priori no-informativas e informativas. En particular, las distribuciones a priori de Evidencia Minimax (Good, 1968), las de Máxima Información de los Datos (Zellner, 1971) y las de Referencia (Bernardo, 1979) son vistas como casos especiales de la maximización de un criterio más general. Bajo un enfoque unificador se presentan las distribuciones a priori de Good-Bemardo-Zellner, que aplicamos en varios métodos de inferencia Bayesiana útiles en investigación económica. Asimismo, utilizamos las distribuciones de Good-Bernardo-Zellner en varios modelos económicos.
Bayesian Analysis, 2012
Using a collection of simulated and real benchmarks, we compare Bayesian and frequentist regularization approaches under a low informative constraint when the number of variables is almost equal to the number of observations on simulated and real datasets. This comparison includes new global noninformative approaches for Bayesian variable selection built on Zellner's g-priors that are similar to Liang et al. (2008). The interest of those calibration-free proposals is discussed. The numerical experiments we present highlight the appeal of Bayesian regularization methods, when compared with non-Bayesian alternatives. They dominate frequentist methods in the sense that they provide smaller prediction errors while selecting the most relevant variables in a parsimonious way.
Mixtures of -priors for Bayesian model averaging with economic applications
Journal of Econometrics, 2012
This paper examines the issue of variable selection in linear regression modeling, where there is a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset. In this context, Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. The main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. The authors combine a Binomial-Beta prior on model size with a g-prior on the coefficients of each model. In addition, they assign a hyperprior to g, as the choice of g has been found to have a large impact on the . The authors may be contacted at M.F.Steel@stats.warwick.ac.uk and eley@worldbank.org .
Prior Distributions for Objective Bayesian Analysis
Bayesian Analysis
We provide a review of prior distributions for objective Bayesian analysis. We start by examining some foundational issues and then organize our exposition into priors for: i) estimation or prediction; ii) model selection; iii) highdimensional models. With regard to i), we present some basic notions, and then move to more recent contributions on discrete parameter space, hierarchical models, nonparametric models, and penalizing complexity priors. Point ii) is the focus of this paper: it discusses principles for objective Bayesian model comparison, and singles out some major concepts for building priors, which are subsequently illustrated in some detail for the classic problem of variable selection in normal linear models. We also present some recent contributions in the area of objective priors on model space. With regard to point iii) we only provide a short summary of some default priors for high-dimensional models, a rapidly growing area of research.
Mixtures of g-priors for Bayesian model averaging with economic applications
2010
This paper examines the issue of variable selection in linear regression modeling, where there is a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset. In this context, Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. The main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. The authors combine a Binomial-Beta prior on model size with a g-prior on the coefficients of each model. In addition, they assign a hyperprior to g, as the choice of g has been found to have a large impact on the . The authors may be contacted at M.F.Steel@stats.warwick.ac.uk and eley@worldbank.org .
A Note on Bayesian Prediction from the Regression Model with Informative Priors
Australian <html_ent glyph="@amp;" ascii="&"/> New Zealand Journal of Statistics, 2001
This paper considers the problem of undertaking a predictive analysis from a regression model when proper conjugate priors are used. It shows how the prior information can be incorporated as a virtual experiment by augmenting the data, and it derives expressions for both the prior and the posterior predictive densities. The results obtained are of considerable practical importance to practitioners of Bayesian regression methods.
Benchmark priors for Bayesian Model averaging
Journal of Econometrics, 1998
In contrast to a posterior analysis given a particular sampling model, posterior model probabilities in the context of model uncertainty are typically rather sensitive to the specification of the prior. In particular, "diffuse" priors on model-specific parameters can lead to quite unexpected consequences. Here we focus on the practically relevant situation where we need to entertain a (large) number of sampling models and we have (or wish to use) little or no subjective prior information. We aim at providing an "automatic" or "benchmark" prior structure that can be used in such cases.
ON DUAL EXPRESSION OF PRIOR INFORMATION IN BAYESIAN PARAMETER ESTIMATION
In Bayesian parameter estimation, a priori information can be used to shape the prior density of unknown parameters of the model. When chosen in a conjugate, selfreproducing form, the prior density of parameters is nothing but a model-based transform of a certain "prior" density of observed data. This observation suggests two possible ways of expressing a priori knowledge-in terms of parameters of a particular model and in terms of data entering the model. The latter way turns out useful when dealing with statistical models whose parameters lack a direct physical interpretation. In practice, the amount of a priori information is usually not sufficient for complete specification of the prior density of data. The paper shows an information-based way of converting such incomplete information into the prior density of unknown parameters.
Marginally specified priors for non-parametric Bayesian estimation
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2014
Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Additionally, posterior approximations under these informative priors can generally be made via minor adjustments to existing Markov chain approximation algorithms for standard nonparametric prior distributions. We illustrate the use of such priors in the context of multivariate density estimation using Dirichlet process mixture models, and in the modeling of high-dimensional sparse contingency tables.