Definitions, methods, and applications in interpretable machine learning - PubMed (original) (raw)

Definitions, methods, and applications in interpretable machine learning

W James Murdoch et al. Proc Natl Acad Sci U S A. 2019.

Abstract

Machine-learning models have demonstrated great success in learning complex patterns that enable them to make predictions about unobserved data. In addition to using models for prediction, the ability to interpret what a model has learned is receiving an increasing amount of attention. However, this increased focus has led to considerable confusion about the notion of interpretability. In particular, it is unclear how the wide array of proposed interpretation methods are related and what common concepts can be used to evaluate them. We aim to address these concerns by defining interpretability in the context of machine learning and introducing the predictive, descriptive, relevant (PDR) framework for discussing interpretations. The PDR framework provides 3 overarching desiderata for evaluation: predictive accuracy, descriptive accuracy, and relevancy, with relevancy judged relative to a human audience. Moreover, to help manage the deluge of interpretation methods, we introduce a categorization of existing techniques into model-based and post hoc categories, with subgroups including sparsity, modularity, and simulatability. To demonstrate how practitioners can use the PDR framework to evaluate and understand interpretations, we provide numerous real-world examples. These examples highlight the often underappreciated role played by human audiences in discussions of interpretability. Finally, based on our framework, we discuss limitations of existing methods and directions for future work. We hope that this work will provide a common vocabulary that will make it easier for both practitioners and researchers to discuss and choose from the full range of interpretation methods.

Keywords: explainability; interpretability; machine learning; relevancy.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interest.

Figures

Fig. 1.

Fig. 1.

Overview of different stages (black text) in a data–science life cycle where interpretability is important. Main stages are discussed in Section 3 and accuracy (blue text) is described in Section 4.

Fig. 2.

Fig. 2.

Impact of interpretability methods on descriptive and predictive accuracies. Model-based interpretability (Section 5) involves using a simpler model to fit the data which can negatively affect predictive accuracy, but yields higher descriptive accuracy. Post hoc interpretability (Section 6) involves using methods to extract information from a trained model (with no effect on predictive accuracy). These correspond to the model and post hoc stages in Fig. 1.

Similar articles

Cited by

References

    1. Litjens G., et al. , A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). - PubMed
    1. Brennan T., Oliver W. L., The emergence of machine learning techniques in criminology. Criminol. Public Policy 12, 551–562 (2013).
    1. Angermueller C., Pärnamaa T., Parts L., Stegle O., Deep learning for computational biology. Mol. Syst. Biol. 12, 878 (2016). - PMC - PubMed
    1. Vu M. A. T., et al. , A shared vision for machine learning in neuroscience. J. Neurosci. 38, 1601–1607 (2018). - PMC - PubMed
    1. Goodman B., Flaxman S., European Union regulations on algorithmic decision-making and a “right to explanation”. arXiv:1606.08813 (31 August 2016).

Publication types

LinkOut - more resources