Bayesian Model Selection and Model Averaging (original) (raw)
Related papers
Objective Bayesian Methods for Model Selection: Introduction and Comparison
Institute of Mathematical Statistics Lecture Notes - Monograph Series, 2001
The basics of the Bayesian approach to model selection are first presented, as well as the motivations for the Bayesian approach. We then review four methods of developing default Bayesian procedures that have undergone considerable recent development, the Conventional Prior approach, the Bayes Information Criterion, the Intrinsic Bayes Factor, and the Fractional Bayes Factor. As part of the review, these methods are illustrated on examples involving the normal linear model. The later part of the chapter focuses on comparison of the four approaches, and includes an extensive discussion of criteria for judging model selection procedures.
The practical implementation of Bayesian model selection
2001
In principle, the Bayesian approach to model selection is straightforward. Prior probability distributions are used to describe the uncertainty surrounding all unknowns. After observing the data, the posterior distribution provides a coherent post data summary of the remaining uncertainty which is relevant for model selection. However, the practical implementation of this approach often requires carefully tailored priors and novel posterior calculation methods. In this article, we illustrate some of the fundamental practical issues that arise for two different model selection problems: the variable selection problem for the linear model and the CART model selection problem.
Bayesian model selection and averaging via mixture model estimation
2018
A new approach for Bayesian model averaging (BMA) and selection is proposed, based on the mixture model approach for hypothesis testing in [12]. Inheriting from the good properties of this approach, it extends BMA to cases where improper priors are chosen for parameters that are common to all candidate models. From an algorithmic point of view, our approach consists in sampling from the posterior distribution of the single-datum mixture of all candidate models, weighted by their prior probabilities. We show that this posterior distribution is equal to the ‘Bayesian-model averaged’ posterior distribution over all candidate models, weighted by their posterior probability. From this BMA posterior sample, a simple Monte-Carlo estimate of each model’s posterior probability is derived, as well as importance sampling estimates for expectations under each model’s posterior distribution.
Bayesian Model Averaging and Model Search Strategies
1998
In regression models, such as generalized linear models, there is often substantial prior uncertainty about the choice of covariates to include. Conceptually, the Bayesian paradigm can easily incorporate this form of model uncertainty by building an expanded model that includes all possible subsets of covariates. In Bayesian model averaging, predictive distributions or posterior distributions of quantities of interest are obtained as mixtures of the model-specific distributions weighted by the posterior model probabilities. A major difficulty in implementing this approach is that the number of models in the mixture is often so large that enumeration of all models is impossible and some type of search strategy is required to determine a subset of models to use. In the case of an orthonormal design, some computationally simple approximations to the posterior model probabilities are introduced. These are used to develop efficient methods for deterministic or stochastic sampling from high-dimensional model spaces.
Model selection: Full Bayesian approach
Environmetrics, 2001
We show how the Full Bayesian Signi®cance Test (FBST) can be used as a model selection criterion. The FBST was presented in Pereira and Stern as a coherent Bayesian signi®cance test.
arXiv: Computation, 2016
In this paper we briefly review the main methodological aspects concerned with the application of the Bayesian approach to model choice and model averaging in the context of variable selection in regression models. This includes prior elicitation, summaries of the posterior distribution and computational strategies. We then examine and compare various publicly available {\tt R}-packages for its practical implementation summarizing and explaining the differences between packages and giving recommendations for applied users. We find that all packages reviewed lead to very similar results, but there are potentially important differences in flexibility and efficiency of the packages.
Methods and Tools for Bayesian Variable Selection and Model Averaging in Normal Linear Regression
International Statistical Review, 2018
This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving. A note on versions: The version presented here may differ from the published version or, version of record, if you wish to cite this item you are advised to consult the publisher's version. Please see the 'permanent WRAP url' above for details on accessing the published version and note that access may require a subscription.
Clustered Bayesian Model Averaging
Bayesian Analysis, 2013
It is sometimes preferable to conduct statistical analyses based on the combination of several models rather than on the selection of a single model, thus taking into account the uncertainty about the true model. Models are usually combined using constant weights that do not distinguish between different regions of the covariate space. However, a procedure that performs well in a given situation may not do so in another situation. In this paper, we propose the concept of local Bayes factors, where we calculate the Bayes factors by restricting the models to regions of the covariate space. The covariate space is split in such a way that the relative model efficiencies of the various Bayesian models are about the same in the same region while differing in different regions. An algorithm for clustered Bayes averaging is then proposed for model combination, where local Bayes factors are used to guide the weighting of the Bayesian models. Simulations and real data studies show that clustered Bayesian averaging results in better predictive performance compared to a single Bayesian model or Bayesian model averaging where models are combined using the same weights over the entire covariate space.