Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies | American Political Science Review | Cambridge Core (original) (raw)

Abstract

Identifying causal mechanisms is a fundamental goal of social science. Researchers seek to study not only whether one variable affects another but also how such a causal relationship arises. Yet commonly used statistical methods for identifying causal mechanisms rely upon untestable assumptions and are often inappropriate even under those assumptions. Randomizing treatment and intermediate variables is also insufficient. Despite these difficulties, the study of causal mechanisms is too important to abandon. We make three contributions to improve research on causal mechanisms. First, we present a minimum set of assumptions required under standard designs of experimental and observational studies and develop a general algorithm for estimating causal mediation effects. Second, we provide a method for assessing the sensitivity of conclusions to potential violations of a key assumption. Third, we offer alternative research designs for identifying causal mechanisms under weaker assumptions. The proposed approach is illustrated using media framing experiments and incumbency advantage studies.

References

Angrist, J. D., Imbens, G. W., and Rubin, D. B.. 1996. “Identification of Causal Effects Using Instrumental Variables (with Discussion).” Journal of the American Statistical Association 91 (434): 444–55.CrossRefGoogle Scholar

Ansolabehere, S., Snowberg, E. C., and Snyder, J. M.. 2006. “Television and the Incumbency Advantage in U.S. Elections.” Legislative Studies Quarterly 31 (4): 469–90.CrossRefGoogle Scholar

Ansolabehere, S., Snyder, J. M., and Stewart, C.. 2000. “Old Voters, New Voters, and the Personal Vote: Using Redistricting to Measure the Incumbency Advantage.” American Journal of Political Science 44 (1): 17–34.CrossRefGoogle Scholar

Baron, R. M., and Kenny, D. A.. 1986. “The Moderator–Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51 (6): 1173–82.CrossRefGoogle ScholarPubMed

Bartels, L. M. 1993. “Messages Received: The Political Impact of Media Exposure.” American Political Science Review 87 (2): 267–85.CrossRefGoogle Scholar

Bertrand, M., and Mullainathan, S.. 2004. “Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination.” American Economic Review 94 (4): 991–1013.CrossRefGoogle Scholar

Blattman, C. 2009. “From Violence to Voting: War and Political Participation in Uganda.” American Political Science Review 103 (2): 231–47.CrossRefGoogle Scholar

Brader, T., Valentino, N. A., and Suhay, E.. 2008. “What Triggers Public Opposition to Immigration? Anxiety, Group Cues, and Immigration.” American Journal of Political Science 52 (4): 959–78.CrossRefGoogle Scholar

Brady, H. E., and Collier, D.. 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, MD: Rowman and Littlefield.Google Scholar

Bullock, J., Green, D., and Ha, S.. 2010. “Yes, But What's the Mechanism? (Don't Expect an Easy Answer).” Journal of Personality and Social Psychology 98 (4): 550–58.CrossRefGoogle ScholarPubMed

Chong, D., and Druckman, J.. 2007. “Framing Theory.” Annual Review of Political Science 10: 103–26.CrossRefGoogle Scholar

Cnudde, C. F., and McCrone, D. J.. 1966. “The Linkage between Constituency Attitudes and Congressional Voting Behavior: A Causal Model.” American Political Science Review 60 (1): 66–72.CrossRefGoogle Scholar

Cole, S. R., and Frangakis, C. E.. 2009. “The Consistency Statement in Causal Inference: A Definition or Assumption?” Epidemiology 20 (1): 3–5.CrossRefGoogle ScholarPubMed

Collier, D., Brady, H. E., and Seawright, J.. 2004. “Source of Leverage in Causal Inference: Toward an Alternative View of Methodology.” In Rethinking Social Inquiry:Diverse Tools, Shared Standards eds. Brady, H. and Collier, D., Berkeley, CA: Rowman and Littlefield.Google Scholar

Cox, G. W., and Katz, J. N.. 1996. “Why Did the Incumbency Advantage in U.S. House Elections Grow?” American Journal of Political Science 40 (2): 478–97.CrossRefGoogle Scholar

Deaton, A. 2010a. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.CrossRefGoogle Scholar

Deaton, A. 2010b. “Understanding the Mechanisms of Economic Development.” Journal of Economic Perspectives 24 (3): 3–16.CrossRefGoogle Scholar

Druckman, J. 2005. “Media Matter: How Newspapers and Television News Cover Campaigns and Influence Voters.” American Political Science Review 22: 463–81.Google Scholar

Erikson, R. S., and Palfrey, T. R.. 1998. “Campaign Spending and Incumbency: An Alternative Simultaneous Equations Approach.” Journal of Politics 60 (2): 355–73.CrossRefGoogle Scholar

Gadarian, S. K. 2010. “The Politics of Threat: How Terrorism News Shapes Foreign Policy Attitudes.” Journal of Politics 72 (2): 469–83.CrossRefGoogle Scholar

Gelman, A., and King, G.. 1990. “Estimating Incumbency Advantage without Bias.” American Journal of Political Science 34 (4): 1142–64.CrossRefGoogle Scholar

Gerber, A. 1998. “Estimating the Effect of Campaign Spending on Senate Election Outcomes Using Instrumental Variables.” American Political Science Review 92 (2): 401–11.CrossRefGoogle Scholar

Glynn, A. N. 2010. “The Product and Difference Fallacies for Indirect Effects.” Department of Government, Harvard University. Unpublished manuscript, Mimeo.Google Scholar

Green, D. P., Ha, S. E., and Bullock, J. G.. 2010. “Enough Already about Black Box Experiments: Studying Mediation Is More Difficult Than Most Scholars Suppose.” Annals of the American Academy of Political and Social Sciences 628 (1): 200–08.CrossRefGoogle Scholar

Gross, J. J., and Levenson, R. W.. 1995. “Eliciting Emotions Using Films.” Cognition and Emotion 9 (1): 87–108.CrossRefGoogle Scholar

Haavelmo, T. 1943. “The Statistical Implications of a System of Simultaneous Equations.” Econometrica 11: 1–12.CrossRefGoogle Scholar

Heckman, J. J., and Smith, J. A.. 1995. “Assessing the Case for Social Experiments.” Journal of Economic Perspectives 9 (2): 85–110.CrossRefGoogle Scholar

Hetherington, M. J. 2001. “Resurgent Mass Partisanship: The Role of Elite Polarization.” American Political Science Review 95 (3): 619–31.CrossRefGoogle Scholar

Ho, D. E., Imai, K., King, G., and Stuart, E. A.. 2007. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.” Political Analysis 15 (3): 199–236.CrossRefGoogle Scholar

Holland, P. W. 1986. “Statistics and Causal Inference.” Journal of the American Statistical Association 81: 945–60.CrossRefGoogle Scholar

Holland, P. W. 1988. “Causal Inference, Path Analysis, and Recursive Structural Equations Models.” Sociological Methodology 18: 449–84.CrossRefGoogle Scholar

Horiuchi, Y., Imai, K., and Taniguchi, N.. 2007. “Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment.” American Journal of Political Science 51 (3): 669–87.CrossRefGoogle Scholar

Imai, K., Keele, L., and Tingley, D.. 2010. “A General Approach to Causal Mediation Analysis.” Psychological Methods 15 (4): 309–34.CrossRefGoogle ScholarPubMed

Imai, K., Keele, L., Tingley, D. and Yamamoto, T.. 2010. “Causal Mediation Analysis Using R”. In Advances in Social Science Research Using R, ed. Vinod, H. D., Lecture Notes in Statistics. Springer-verlag: New York, 129–54.CrossRefGoogle Scholar

Imai, K., Keele, L., Tingley, D., and Yamamoto, T.. 2011. “Replication Data for: Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” The Dataverse Network. hdl:1902.1/16467 (accessed September 1, 2011).Google Scholar

Imai, K., Keele, L., and Yamamoto, T.. 2010. “Identification, Inference, and Sensitivity Analysis for Causal Mediation Effects.” Statistical Science 25 (1): 51–71.CrossRefGoogle Scholar

Imai, K., King, G., and Stuart, E. A.. 2008. “Misunderstandings among Experimentalists and Observationalists about Causal Inference.” Journal of the Royal Statistical Society, Series A (Statistics in Society) 171 (2): 481–502.CrossRefGoogle Scholar

Imai, K., and Tingley, D.. N.d. “A Statistical Method for Empirical Testing of Competing Theories.” American Journal of Political Science. Forthcoming.Google Scholar

Imai, K., Tingley, D., and Yamamoto, T.. N.d. “Experimental Designs for Identifying Causal Mechanisms.” (With discussions). Journal of the Royal Statistical Society, Series A (Statistics in Society). Forthcoming.Google Scholar

Imai, K., and Yamamoto, T.. 2010. “Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis.” American Journal of Political Science 54 (2): 543–60.CrossRefGoogle Scholar

Isbell, L., and Ottati, V.. 2002. “The Emotional Voter.” In The Social Psychology of Politics, ed. Ottati, V., New York: Kluwer, 55–74.CrossRefGoogle Scholar

Jacobson, G. C. 1987. “The Politics of Congressional Elections. Boston: Little, Brown.Google Scholar

Jo, B. 2008. “Causal Inference in Randomized Experiments with Mediational Processes.” Psychological Methods 13 (4): 314–36.CrossRefGoogle ScholarPubMed

Jost, J. T., Napier, J. L., Thorisdottir, H., Gosling, S. D., Palfai, T. P., and Ostafin, B.. 2007. “Are Needs to Manage Uncertainty and Threat Associated With Political Conservatism or Ideological Extremity?” Personality and Social Psychology Bulletin 33 (7): 989–1007.CrossRefGoogle ScholarPubMed

Kinder, D. R. and Sanders, L.. 1996. Divided by Color: Racial Politics and Democratic Ideals. Chicago: University of Chicago Press.Google Scholar

King, G., Keohane, R. O., and Verba, S.. 1994. Designing Social Inquiry. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar

King, G., Tomz, M., and Wittenberg, J.. 2000. “Making the Most of Statistical Analyses: Improving Interpretation and Presentation.” American Journal of Political Science 44: 341–55.CrossRefGoogle Scholar

Levitt, S. D. and Wolfram, C. D.. 1997. “Decomposing the Sources of Incumbency Advantage in the U.S. House.” Legislative Studies Quarterly 22 (1): 45–60.CrossRefGoogle Scholar

MacKinnon, D. 2008. Introduction to Statistical Mediation Analysis. New York: Routledge.Google Scholar

MacKinnon, D., Lockwood, C., Brown, C., Wang, W., and Hoffman, J.. 2007. “The Intermediate Endpoint Effect in Logistic and Probit Regression.” Clinical Trials 4: 499–513.CrossRefGoogle ScholarPubMed

Manski, C. F. 2007. Identification for Prediction and Decision. Cambridge, MA: Harvard University Press.Google Scholar

Miller, J. M., and Krosnick, J. A.. 2000. “News Media Impact on the Ingredients of Presidential Evaluations: Politically Knowledgeable Citizens Are Guided by a Trusted Source.” American Journal of Political Science 44 (2): 301–15.CrossRefGoogle Scholar

Miller, W. E., and Stokes, D. W.. 1963. “Constituency Influence in Congress.” American Political Science Review 57 (1): 45–46.CrossRefGoogle Scholar

Nelson, T. E., Clawson, R. A., and Oxley, Z. M.. 1997. “Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance.” American Political Science Review 91 (3): 567–83.CrossRefGoogle Scholar

Nelson, T. E., and Kinder, D. R.. 1996. “Issue Frames and Group-centrism in American Public Opinion.” The Journal of Politics 58 (4): 1055–78.CrossRefGoogle Scholar

Neyman, J. [1923] 1990. “On the Application of Probability Theory to Agricultural Experiments: Essay on Principles, Section 9.” Statistical Science 5: 465–80.Google Scholar

Olsson, A., Ebert, J. P., Banaji, M. R., and Phelps, E. A.. 2005. “The Role of Social Groups in the Persistence of Learned Fear.” Science 309 (5735): 785–87.CrossRefGoogle ScholarPubMed

Oxley, D. R., Smith, K. B., Alford, J. R., Hibbing, M. V., Miller, J. L., Scalora, M., Hatemi, P. K., and Hibbing, J. R.. 2008. “Political Attitudes Vary with Physiological Traits.” Science 321 (5896): 1667–70.CrossRefGoogle ScholarPubMed

Pearl, J. 2001. “Direct and Indirect Effects.” In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, eds. Breese, Jack S. and Koller, Daphne. San Francisco: Morgan Kaufmann, 411–20.Google Scholar

Pearl, J. N.d. “The Causal Mediation Formula: A Guide to the Assessment of Pathways and Mechanisms.” Prevention Science. Forthcoming.Google Scholar

Petersen, M. L., Sinisi, S. E., and van der Laan, M. J.. 2006. “Estimation of Direct Causal Effects.” Epidemiology 17 (3): 276–84.CrossRefGoogle ScholarPubMed

Prior, M. 2006. “The Incumbent in the Living Room: The Rise of Television and the Incumbency Advantage in U.S. House Elections.” Journal of Politics 68 (3): 657–73.CrossRefGoogle Scholar

Robins, J. M. 2003. “Semantics of Causal DAG Models and the Identification of Direct and Indirect Effects.” In Highly Structured Stochastic Systems, eds. Green, P. J., Hjort, N. L., and Richardson, S., Oxford: Oxford University Press, 70–81.CrossRefGoogle Scholar

Robins, J. M., and Greenland, S.. 1992. “Identifiability and Exchangeability for Direct and Indirect Effects.” Epidemiology 3 (2): 143–55.CrossRefGoogle ScholarPubMed

Robins, J. M., and Richardson, T.. 2010. “Alternative Graphical Causal Models and the Identification of Direct Effects. In Causality and Psychopathology: Finding the Determinants of Disorders and Their Cures, eds. Shrout, P., Keyes, K., and Omstein, K., Oxford: Oxford University Press, 103–58.Google Scholar

Rosenbaum, P. R. 2002a. “Covariance Adjustment in Randomized Experiments and Observational Studies: Rejoinder.” Statistical Science 17 (3): 321–27.CrossRefGoogle Scholar

Rosenbaum, P. R. 2002b. “Covariance Adjustment in Randomized Experiments and Observational Studies (with Discussion).” Statistical Science 17 (3): 286–327.CrossRefGoogle Scholar

Rubin, D. B. 1974. “Estimating Causal Effects of Treatments in Randomized and Non-randomized Studies.” Journal of Educational Psychology 66: 688–701.CrossRefGoogle Scholar

Shadish, W. R., Cook, T. D., and Campbell, D. T.. 2001. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.Google Scholar

Sjölander, A. 2009. “Bounds on Natural Direct Effects in the Presence of Confounded Intermediate Variables.” Statistics in Medicine 28 (4): 558–71.CrossRefGoogle ScholarPubMed

Sobel, M. E. 1982. “Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models.” Sociological Methodology 13: 290–321.CrossRefGoogle Scholar

Sobel, M. E. 2008. “Identification of Causal Parameters in Randomized Studies with Mediating Variables.” Journal of Educational and Behavioral Statistics 33 (2): 230–51.CrossRefGoogle Scholar

Spencer, S., Zanna, M., and Fong, G.. 2005. “Establishing a Causal Chain: Why Experiments Are Often More Effective Than Mediational Analyses in Examining Psychological Processes.” Journal of Personality and Social Psychology 89 (6): 845–51.CrossRefGoogle ScholarPubMed

Tiedens, L. Z. and Linton, S.. 2001. “Judgment under Emotional Certainty and Uncertainty: The Effects of Specific Emotions on Information Processing.” Journal of Personality and Social Psychology 81 (6): 973–88.CrossRefGoogle ScholarPubMed

Tomz, M. and Houweling, R. P. van. 2009. “The Electoral Implications of Candidate Ambiguity.” American Political Science Review 103 (1): 83–98.CrossRefGoogle Scholar

VanderWeele, T. J. 2009. “Marginal Structural Models for the Estimation of Direct and Indirect Effects.” Epidemiology 20 (1): 18–26.CrossRefGoogle ScholarPubMed

VanderWeele, T. J. and Robins, J. M.. 2009. “Minimal Sufficient Causation and Directed Acyclic Graphs.” Annals of Statistics 37 (3): 1437–65.CrossRefGoogle Scholar