Choice of Priors and Variable Selection in Bayesian Regression (original) (raw)
Related papers
Bayesian Variable Selection in Linear Regression and a Comparison
Hacettepe Journal of Mathematics and Statistics, 2001
In this study, Bayesian approaches, such as Zellner, Occam's Window and Gibbs sampling, have been compared in terms of selecting the correct subset for the variable selection in a linear regression model. The aim of this comparison is to analyze Bayesian variable selection and the behavior of classical criteria by taking into consideration the different values of β and σ and prior expected levels.
Methods and Tools for Bayesian Variable Selection and Model Averaging in Normal Linear Regression
International Statistical Review, 2018
This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving. A note on versions: The version presented here may differ from the published version or, version of record, if you wish to cite this item you are advised to consult the publisher's version. Please see the 'permanent WRAP url' above for details on accessing the published version and note that access may require a subscription.
Criteria for Bayesian model choice with application to variable selection
The Annals of Statistics, 2012
In objective Bayesian model selection, no single criterion has emerged as dominant in defining objective prior distributions. Indeed, many criteria have been separately proposed and utilized to propose differing prior choices. We first formalize the most general and compelling of the various criteria that have been suggested, together with a new criterion. We then illustrate the potential of these criteria in determining objective model selection priors by considering their application to the problem of variable selection in normal linear models. This results in a new model selection objective prior with a number of compelling properties.
Comparison of Bayesian objective procedures for variable selection in linear regression
TEST, 2008
In the objective Bayesian approach to variable selection in regression a crucial point is the encompassing of the underlying nonnested linear models. Once the models have been encompassed one can define objective priors for the multiple testing problem involved in the variable selection problem. There are two natural ways of encompassing: one way is to encompass all models into the model containing all possible regressors, and the other one is to encompass the model containing the intercept only into any other. In this paper we compare the variable selection procedures that result from each of the two mentioned ways of encompassing by analysing their theoretical properties and their behavior in simulated and real data. Relations with frequentist criteria for model selection such as those based on the R 2 adj , and Mallows C p are provided incidentally.
A novel Bayesian approach for variable selection in linear regression models
Computational Statistics & Data Analysis
We propose a novel Bayesian approach to the problem of variable selection in multiple linear regression models. In particular, we present a hierarchical setting which allows for direct specification of a-priori beliefs about the number of nonzero regression coefficients as well as a specification of beliefs that given coefficients are nonzero. To guarantee numerical stability, we adopt a g-prior with an additional ridge parameter for the unknown regression coefficients. In order to simulate from the joint posterior distribution an intelligent random walk Metropolis-Hastings algorithm which is able to switch between different models is proposed. Testing our algorithm on real and simulated data illustrates that it performs at least on par and often even better than other well-established methods. Finally, we prove that under some nominal assumptions, the presented approach is consistent in terms of model selection.
arXiv: Computation, 2016
In this paper we briefly review the main methodological aspects concerned with the application of the Bayesian approach to model choice and model averaging in the context of variable selection in regression models. This includes prior elicitation, summaries of the posterior distribution and computational strategies. We then examine and compare various publicly available {\tt R}-packages for its practical implementation summarizing and explaining the differences between packages and giving recommendations for applied users. We find that all packages reviewed lead to very similar results, but there are potentially important differences in flexibility and efficiency of the packages.
On consistency of Bayesian variable selection procedures
2012
In this paper we extend the pairwise consistency of the Bayesian procedure to the entire class of linear models when the number of regressors grows as thesample size grows, and it is seen that for establishing consistency both the prior overthe model parameters and the prior over the models play now an important role. Wewill show that commonly used Bayesian procedures with non–fully Bayes priors formodels and for model parameters are inconsistent, and that fully Bayes versions ofthese priors correct this undesirable behavior.
Prior Distributions for Objective Bayesian Analysis
Bayesian Analysis
We provide a review of prior distributions for objective Bayesian analysis. We start by examining some foundational issues and then organize our exposition into priors for: i) estimation or prediction; ii) model selection; iii) highdimensional models. With regard to i), we present some basic notions, and then move to more recent contributions on discrete parameter space, hierarchical models, nonparametric models, and penalizing complexity priors. Point ii) is the focus of this paper: it discusses principles for objective Bayesian model comparison, and singles out some major concepts for building priors, which are subsequently illustrated in some detail for the classic problem of variable selection in normal linear models. We also present some recent contributions in the area of objective priors on model space. With regard to point iii) we only provide a short summary of some default priors for high-dimensional models, a rapidly growing area of research.
Approaches for Bayesian variable selection
1997
This paper describes and compares various hierarchical mixture prior formulations of variable selection uncertainty in normal linear regression models. These include the nonconjugate SSVS formulation of George and McCulloch (1993), as well as conjugate formulations which allow for analytical simplification. Hyperparameter settings which base selection on practical significance, and the implications of using mixtures with point priors are discussed. Computational methods for posterior evaluation and exploration are considered. Rapid updating methods are seen to provide feasible methods for exhaustive evaluation using Gray Code sequencing in moderately sized problems, and fast Markov Chain Monte Carlo exploration in large problems. Estimation of normalization constants is seen to provide improved posterior estimates of individual model probabilities and the total visited probability. Various procedures are illustrated on simulated sample problems and on a real problem concerning the construction of financial index tracking portfolios.
The power-conditional-expected-posterior (PCEP) prior developed for variable selection in normal regression models combines ideas from the power-prior and expected-posterior prior, relying on the concept of random imaginary data, and provides a consistent variable selection method which leads to parsimonious inference. In this paper we discuss the computational limitations of applying the PCEP prior to generalized linear models (GLMs) and present two PCEP prior variations which are easily applicable to regression models belonging to the exponential family of distributions. We highlight the differences between the initial PCEP prior and the two GLM-based PCEP prior adaptations and compare their properties in the conjugate case of the normal linear regression model. Hyper prior extensions for the PCEP power parameter are further considered. We consider several simulation scenarios and one real data example for evaluating the performance of the proposed methods compared to other common...