The Logic of the Survey Experiment Reexamined | Political Analysis | Cambridge Core (original) (raw)

Abstract

Scholars of political behavior increasingly embed experimental designs in opinion surveys by randomly assigning respondents alternative versions of questionnaire items. Such experiments have major advantages: they are simple to implement and they dodge some of the difficulties of making inferences from conventional survey data. But survey experiments are no panacea. We identify problems of inference associated with typical uses of survey experiments in political science and highlight a range of difficulties, some of which have straightforward solutions within the survey-experimental approach and some of which can be dealt with only by exercising greater caution in interpreting findings and bringing to bear alternative strategies of research.

References

Achen, Christopher H. 1986. The statistical analysis of quasi-experiments. Berkeley, CA: University of California Press.CrossRefGoogle Scholar

Ansolabehere, Stephen, and Iyengar, Shanto. 1995. Going negative: How attack ads shrink and polarize the electorate. New York: Free Press.Google Scholar

Brady, Henry E., and Collier, David, eds. 2000. Rethinking social inquiry: Diverse tools, shared standards. Lanham, MD: Rowman and Littlefield.Google Scholar

Clarke, Harold D., Kornberg, Allan, McIntyre, Chris, Bauer-Kaase, Petra, and Kaase, Max. 1999. The effect of economic priorities on the measurement of value change: New experimental evidence. American Political Science Review 93: 637–47.CrossRefGoogle Scholar

Davis, Darren W., and Silver, Brian D. 2003. Stereotype threat and race of interviewer effects in a survey on political knowledge. American Journal of Political Science 47: 33–45.Google Scholar

Druckman, James N. 2001. The implications of framing effects for citizen competence. Political Behavior 23: 225–56.CrossRefGoogle Scholar

Druckman, James N., and Lupia, Arthur. 2006. Mind, will, and choice. In The Oxford handbook on contextual political analysis, ed. Tilly, Charles and Goodwin, Robert E., 97–113. Oxford: Oxford University Press.Google Scholar

Druckman, James N., and Nelson, Kjersten R. 2003. Framing and deliberation: How citizens' conversations limit elite influence. American Journal of Political Science 47: 729–45.CrossRef. American Journal of Political Science 41: 30–60.Google Scholar

Rosenbaum, Paul R., and Rubin, Donald B. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70: 41–55.CrossRefGoogle Scholar

Schuman, Howard, and Bobo, Lawrence. 1988. Survey-based experiments on white racial attitudes toward residential segregation. American Journal of Sociology 94: 273–99.Google Scholar

Smith, Alastair. 1999. Testing theories of strategic choice: The example of crisis escalation. American Journal of Political Science 43: 1254–83.Google Scholar

Sniderman, Paul M. 1996. Innovations in experimental design in attitude surveys. Annual Review of Sociology 22: 377–99.CrossRefGoogle Scholar

Sniderman, Paul M., Brody, Richard A., and Tetlock, Philip E. 1991. Reasoning and choice: Explorations in political psychology. Cambridge: Cambridge University Press.Google Scholar

Sniderman, Paul M., and Carmines, Edward G. 1997. Reaching beyond race. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar

Sniderman, Paul M., and Piazza, Thomas. 1993. The scar of race. Cambridge, MA: Belknap Press.CrossRefGoogle Scholar

Sniderman, Paul M., and Theriault, Sean. 2004. The dynamics of political argument and the logic of issue framing. In Studies in public opinion: Attitudes, nonattitudes, measurement error, and change, ed. Saris, Willem E. and Sniderman, Paul M., 133–65. Princeton, NJ: Princeton University Press.Google Scholar

Sniderman, Paul M., Hagendoorn, Louk, and Prior, Markus. 2004. Predispositional factors and situational triggers: Exclusionary reactions to immigrant minorities. American Political Science Review 98: 35–50.Google Scholar

Taber, Charles S., and Lodge, Milton. 2006. Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science 50: 755–69.Google Scholar

Zaller, John. 1992. The nature and origins of mass opinion. New York: Cambridge University Press.CrossRefGoogle Scholar