Bayesian Model Selection and Model Averaging (original) (raw)
Objective Bayesian Methods for Model Selection: Introduction and Comparison
Institute of Mathematical Statistics Lecture Notes - Monograph Series, 2001
The basics of the Bayesian approach to model selection are first presented, as well as the motivations for the Bayesian approach. We then review four methods of developing default Bayesian procedures that have undergone considerable recent development, the Conventional Prior approach, the Bayes Information Criterion, the Intrinsic Bayes Factor, and the Fractional Bayes Factor. As part of the review, these methods are illustrated on examples involving the normal linear model. The later part of the chapter focuses on comparison of the four approaches, and includes an extensive discussion of criteria for judging model selection procedures.
The practical implementation of Bayesian model selection
2001
In principle, the Bayesian approach to model selection is straightforward. Prior probability distributions are used to describe the uncertainty surrounding all unknowns. After observing the data, the posterior distribution provides a coherent post data summary of the remaining uncertainty which is relevant for model selection. However, the practical implementation of this approach often requires carefully tailored priors and novel posterior calculation methods. In this article, we illustrate some of the fundamental practical issues that arise for two different model selection problems: the variable selection problem for the linear model and the CART model selection problem.
Bayesian model selection and averaging via mixture model estimation
2018
A new approach for Bayesian model averaging (BMA) and selection is proposed, based on the mixture model approach for hypothesis testing in [12]. Inheriting from the good properties of this approach, it extends BMA to cases where improper priors are chosen for parameters that are common to all candidate models. From an algorithmic point of view, our approach consists in sampling from the posterior distribution of the single-datum mixture of all candidate models, weighted by their prior probabilities. We show that this posterior distribution is equal to the ‘Bayesian-model averaged’ posterior distribution over all candidate models, weighted by their posterior probability. From this BMA posterior sample, a simple Monte-Carlo estimate of each model’s posterior probability is derived, as well as importance sampling estimates for expectations under each model’s posterior distribution.
Bayesian Model Averaging and Model Search Strategies
1998
In regression models, such as generalized linear models, there is often substantial prior uncertainty about the choice of covariates to include. Conceptually, the Bayesian paradigm can easily incorporate this form of model uncertainty by building an expanded model that includes all possible subsets of covariates. In Bayesian model averaging, predictive distributions or posterior distributions of quantities of interest are obtained as mixtures of the model-specific distributions weighted by the posterior model probabilities. A major difficulty in implementing this approach is that the number of models in the mixture is often so large that enumeration of all models is impossible and some type of search strategy is required to determine a subset of models to use. In the case of an orthonormal design, some computationally simple approximations to the posterior model probabilities are introduced. These are used to develop efficient methods for deterministic or stochastic sampling from high-dimensional model spaces.
Model selection: Full Bayesian approach
Environmetrics, 2001
We show how the Full Bayesian Signi®cance Test (FBST) can be used as a model selection criterion. The FBST was presented in Pereira and Stern as a coherent Bayesian signi®cance test.
arXiv: Computation, 2016
In this paper we briefly review the main methodological aspects concerned with the application of the Bayesian approach to model choice and model averaging in the context of variable selection in regression models. This includes prior elicitation, summaries of the posterior distribution and computational strategies. We then examine and compare various publicly available {\tt R}-packages for its practical implementation summarizing and explaining the differences between packages and giving recommendations for applied users. We find that all packages reviewed lead to very similar results, but there are potentially important differences in flexibility and efficiency of the packages.
Australian <html_ent glyph="@amp;" ascii="&"/> New Zealand Journal of Statistics, 2006
This article proposes a new data-based prior distribution for the error variance in a Gaussian linear regression model, when the model is used for Bayesian variable selection and model averaging. For a given subset of variables in the model, this prior has a mode that is an unbiased estimator of the error variance but is suitably dispersed to make it uninformative relative to the marginal likelihood. The advantage of this empirical Bayes prior for the error variance is that it is centred and dispersed sensibly and avoids the arbitrary specification of hyperparameters. The performance of the new prior is compared to that of a prior proposed previously in the literature using several simulated examples and two loss functions. For each example our paper also reports results for the model that orthogonalizes the predictor variables before performing subset selection. A real example is also investigated. The empirical results suggest that for both the simulated and real data, the performance of the estimators based on the prior proposed in our article compares favourably with that of a prior used previously in the literature.
Methods and Tools for Bayesian Variable Selection and Model Averaging in Normal Linear Regression
International Statistical Review, 2018
This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving. A note on versions: The version presented here may differ from the published version or, version of record, if you wish to cite this item you are advised to consult the publisher's version. Please see the 'permanent WRAP url' above for details on accessing the published version and note that access may require a subscription.