The Underprovision of Experiments in Political Science (original) (raw)
Related papers
Bayesian analysis in social sciences
Scholarly Community Encyclopedia, 2021
Given the reproducibility crisis (or replication crisis), more psychologists and social-cultural scientists are getting involved with Bayesian inference. Therefore, the current article provides a brief overview of programs (or software) and steps to conduct Bayesian data analysis in social sciences.
Statistical Science, 2011
Fienberg convincingly demonstrates that Bayesian models and methods represent a powerful approach to squeezing illumination from data in public policy settings. However, no school of inference is without its weaknesses, and, in the face of the ambiguities, uncertainties, and poorly posed questions of the real world, perhaps we should not expect to find a formally correct inferential strategy which can be universally applied, whatever the nature of the question: we should not expect to be able to identify a "norm" approach. An analogy is made between George Box's "no models are right, but some are useful," and inferential systems.
On the Empirical Validity of the Bayesian Method
Journal of the Royal Statistical Society: Series B (Methodological), 1993
The ideas in Kolmogorov's programme for algorithmic substantiation of applications of probability make it possible to define a measure of disagreement between the probability distribution representing the attitude of a coherent individual towards a random experiment and the outcome of the experiment. When there is agreement we say that the probability distribution is empirically valid. We prove quantitatively that formulae of Bayesian statistics transform empirically valid probability distributions into other empirically valid distributions.
Statistical inference for experiments
2008
Experiments have become an increasingly common tool for political science researchers over the last decade, particularly laboratory experiments performed on small convenience samples. The standard statistical paradigm used in political science is designed to connect samples to populations, while the inferential goal in an experiment is often about understanding whether an observed treatment effect occurred due to chance. In this paper, we outline an alternative derivation for statistical inference based on randomization of the treatment. While some standard tests approximate this form of inference, many do not and can produce incorrect inferences. These tests also have robust forms that are insensitive to the distribution of the data. We outline common randomization tests, such as Wilcoxon rank tests and the Kruskal-Wallis test, and also develop a randomization test for two-way factorial designs as an alternative to the commonly used two-way ANOVA model. Finally, we reanalyze data from two political science experiments using randomization tests to illustrate the inferential errors that can be made when classical tests are used with data from the lab. 1 Proper design, of course, includes elements other than randomization, such as isolation of the quantity of interest in the manipulated treatment and control over confounders not remedied by randomization, such as maturation or reactivity, perhaps through the use of a proper placebo in a control group, and proper measurement.
Bayesian analyses : where to start and what to report
2014
Most researchers in the social and behavioral sciences will probably have heard of Bayesian statistics in which probability is defined differently compared to classical statistics (probability as the long-run frequency versus probability as the subjective experience of uncertainty). At the same time, many may be unsure of whether they should or would like to use Bayesian methods to answer their research questions (note: all types of conventional questions can also be addressed with Bayesian statistics) . As an attempt to track how popular the methods are, we searched all papers published in 2013 in the field of Psychology (source: Scopus), and we identified 79 empirical papers that used Bayesian methods (see e.g. Dalley, Pollet, & Vidal, 2013; Fife, Weaver, Cool, & Stump, 2013; Ng, Ntoumanis, ThøgersenNtoumani, Stott, & Hindle, 2013). Although this is less than 0.5% of the total number of papers published in this particular field, the fact that ten years ago this number was only 42 ...
Hypothesis evaluation from a Bayesian perspective
Psychological Review, 1983
Bayesian inference provides a general framework for evaluating hypotheses. It is a normative method in the sense of prescribing how hypotheses should be evaluated. However, it may also be used descriptively by characterizing people's actual hypothesis-evaluation behavior in terms of its consistency with or departures from the model. Such a characterization may facilitate the development of psychological accounts of how that behavior is produced. This article explores the potential of Bayesian inference as a theoretical framework for describing how people evaluate hypotheses. First, it identifies a set of logically possible forms of nonBayesian behavior. Second, it reviews existing research in a variety of areas to see whether these possibilities are ever realized. The analysis shows that in some situations several apparently distinct phenomena are usefully viewed as special cases of the same kind of behavior, whereas in other situations previous investigations have conferred a common label (e.g., confirmation bias) to several distinct phenomena. It also calls into question a number of attributions of judgmental bias, suggesting that in some cases the bias is different than what has previously been claimed, whereas in others there may be no bias at all.
Role of Randomization in Bayesian Analysis { an Expository Overview
2005
The evolving role of randomization in Bayesian Analysis as well as arguments for and against randomization is discussed. We note that Bayesian Analysis is moving towards a substantially reduced role for randomization, but it is a sample surveys area where more future work of this kind is needed.
Banter on Bayes: debating the usefulness of Bayesian approaches to solving practical problems
IEEE Expert, 1997
Statistics is the study of uncertainty: how to measure it and what to do about it. Uncertainty itself, as people regard it today, is a fairly new concept in the history of ideas. For example, probability-the branch of mathematics devoted to quantifying uncertainty-dates only from the middle 1600s, brought to life through disputes about how to wager fairly in games of chance. Since then, two main ways to give meaning to the concept of probability have arisen and, to some extent, fought with each other: the frequentist and Bayesian approaches:
Elicited Priors for Bayesian Model Specifications in Political Science Research
Journal of Politics, 2005
We explain how to use elicited priors in Bayesian political science research. These are a form of prior information produced by previous knowledge from structured interviews with subjective area experts who have little or no concern for the statistical aspects of the project. The purpose is to introduce qualitative and area-specific information into an empirical model in a systematic and