Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer - PubMed (original) (raw)
. 2019 Jul;25(7):1054-1056.
doi: 10.1038/s41591-019-0462-y. Epub 2019 Jun 3.
Jakob Nikolas Kather 1 2 3 4 5, Niels Halama 7 8 9, Dirk Jäger 7 10 8, Jeremias Krause 11, Sven H Loosen 11, Alexander Marx 12, Peter Boor 13, Frank Tacke 14, Ulf Peter Neumann 15, Heike I Grabsch 16 17, Takaki Yoshikawa 18 19, Hermann Brenner 7 20 21, Jenny Chang-Claude 22 23, Michael Hoffmeister 20, Christian Trautwein 11, Tom Luedde 24
Affiliations
- PMID: 31160815
- PMCID: PMC7423299
- DOI: 10.1038/s41591-019-0462-y
Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer
Jakob Nikolas Kather et al. Nat Med. 2019 Jul.
Abstract
Microsatellite instability determines whether patients with gastrointestinal cancer respond exceptionally well to immunotherapy. However, in clinical practice, not every patient is tested for MSI, because this requires additional genetic or immunohistochemical tests. Here we show that deep residual learning can predict MSI directly from H&E histology, which is ubiquitously available. This approach has the potential to provide immunotherapy to a much broader subset of patients with gastrointestinal cancer.
Conflict of interest statement
Competing interests
The authors declare no competing interests.
Figures
Extended Data Fig. 1 |. Comparison of five deep neural network architectures.
We compared accuracy and training time of five neural network architectures on the tumor detection dataset with three balanced classes. Alexnet, VGG19 (ref. )) and resnet18 (ref. ) achieved >95% accuracy in withheld images, whereas inceptionv3 (ref. ) and squeezenet had a poor performance on this benchmark task. Among the well-performing models, resnet18 had the lowest number of parameters, making it potentially more portable and less prone to overfitting. In this comparison, we split the dataset into 70% training, 15% validation and 15% test images. Each network is shown twice in this graph: with a learning rate of 1 × 10−6 and 1 × 10−5 (outlined). Training was run for 25 epochs. Resnet18 was subsequently retrained on the dataset, attaining a median fivefold cross-validated out-of-sample AUC > 0.99 for tumor detection. The dataset was derived from n = 94 whole-slide images from n = 81 patients and is available at
https://doi.org/10.5281/zenodo.2530789
.
Extended Data Fig. 2 |. Additional data for classifier performance.
a, Flowchart of all experiments. The area under the receiver operating characteristic curve gives an overall measure of patient-level classifier accuracy as measured in held-out test sets. Flag symbols are from
(licensed under a CC-BY 4.0 license). b, Classification performance in virtual biopsies. We predicted MSI status in all patients in the DACHS cohort, varying the number of blocks (tiles) from 3 to 2,054, which was the median number of blocks per whole-slide image This experiment was repeated five times with different randomly picked blocks being used. As one block has an edge length of 256 μm, a 1-cm tissue cylinder with 100% tumor tissue from a standard 18G biopsy needle corresponds to 117 blocks and a 16G needle corresponds to 156 blocks. In clinical routine, usually only a part of each biopsy core contains tumor, but multiple biopsy cores are collected. With increasing tissue size, performance stabilizes at AUC = 0.84. This shows that a typical biopsy would be sufficient for MSI prediction. CI, confidence interval. c, Distribution of the numbers of blocks for all patients in DACHS (n = 378 patients). d, Overall survival of patients with genetic MSS tumors stratified by high or low predicted MSIness. In this group, patients with high MSIness had a shorter survival than patients with low MSIness. The table shows the number of patients at risk. The P value was calculated by two-sided log-rank test (n = 350 patients).
Extended Data Fig. 3 |. Morphological correlates of intratumor heterogeneity of MSI.
a, Histological image of a test set patient who was genetically determined as MSI. b, Corresponding predicted MSI map for the image shown in a. Three regions are highlighted. Region 1 is a glandular region with necrosis and extracellular mucus; this region was predominantly predicted to be MSS. Region 2 is a solid, dedifferentiated region, which was predicted to be MSI. Region 3 contained mostly budding tumor cells mixed with immune cells, this region was strongly predicted to be MSI. Together, these representative examples show that different morphologies elicit different predictions and that these predictions can be traced back to patterns that are understandable for humans. Scale bar, 2.5 mm. This figure is representative of n = 378 patients in the DACHS cohort.
Extended Data Fig. 4 |. Estimated cost for MSI screening with deep learning.
a, Workflow for MSI screening with deep learning versus immunohistochemistry in tertiary care centers with existing digital pathology core facilities such as the University of Chicago Medical Center. Costs differ by country and are usually cheaper in Europe than in the United States. Here, we list the costs that apply in the United States. b, Set-up cost (fixed cost) for a digital pathology and deep learning infrastructure. H&E, hematoxylin and eosin; MMRd, mismatch repair deficiency; NGS, next-generation sequencing; QC, quality control. Sources and assumptions were as follows. (1) Prices were obtained from
https://htrc.uchicago.edu/fees.php?fee=2&fee=2
, retrieved on 11 March 2019. We assume ×20 magnification on a high-volume whole-slide scanner. (2) Prices were obtained from
https://techcrunch.com/2019/03/07/scaleway-releases-cloud-gpu-instances-for-e1-per-hour/
and
, retrieved on 11 March 2019. We assume that 1 h of GPU computing on a Nvidia Tesla P100 GPU is required to process whole-slide images for one patient to prediction. (3) US Current Procedural Terminology (CPT) code 88342, four-antibody panel at US$852.00 per staining. (4) Personal communication by the Pathology Department, University of Chicago Medicine, March 2019. (5) Personal communication, Medical Oncology, National Center for Tumor Diseases, Germany. (6) Personal experience of cost for a high-throughput slide scanner plus a limited storing capacity, based on offers by multiple digital pathology vendors. (7) Assuming a tower server with one NVidia Tesla V100 GPU or similar GPU, based on multiple offers by providers for professional hardware, March 2019. Staff cost and infrastructure cost are not accounted for in this schematic.
Fig. 1 |. Tumor detection and MSI prediction in H&E histology.
a, A convolutional neural network was trained as a tumor detector for STAD and CRC. Scale bar, 4 mm. b,c, Tumor regions were cut into square tiles (b), which were color-normalized and sorted into MSI and MSS (c). Scale bar, 256 μm. d, Another network was trained to classify MSI versus MSS. e, This automatic pipeline was applied to held-out patient sets.
Fig. 2 |. Classification performance in an external validation set.
a,b, Tissue slides of patients with MSI and MSS tumors in the TCGA-CRC-DX test set show the spatial patterns of predicted MSI score (Extended Data Fig. 4). These images are representative of n = 378 patients. c, A network was trained on the TCGA-CRC-DX training cohort (n = 260 patients) and deployed on the DACHS cohort (n = 378 patients). d, Patient-level receiver operating characteristic curve with bootstrapped 95% CI in DACHS (n = 378 patients). FPR, false-positive rate (1 − specificity); TPR, true-positive rate (sensitivity). e, Pearson correlation of predicted MSIness to transcriptomic and immunohistochemical (IHC) data across test sets. P values are listed in Supplementary Table 4. Sample sizes per cohort are: TCGA-STAD n = 91, TCGA-CRC-KR n = 105, TCGA-CRC-DX n = 95, DACHS n = 134 patients. No adjustments for multiple comparisons were made, and all statistical tests were two-sided.
Comment in
- Advances in artificial intelligence to predict cancer immunotherapy efficacy.
Xie J, Luo X, Deng X, Tang Y, Tian W, Cheng H, Zhang J, Zou Y, Guo Z, Xie X. Xie J, et al. Front Immunol. 2023 Jan 4;13:1076883. doi: 10.3389/fimmu.2022.1076883. eCollection 2022. Front Immunol. 2023. PMID: 36685496 Free PMC article. Review.
Similar articles
- Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study.
Yamashita R, Long J, Longacre T, Peng L, Berry G, Martin B, Higgins J, Rubin DL, Shen J. Yamashita R, et al. Lancet Oncol. 2021 Jan;22(1):132-141. doi: 10.1016/S1470-2045(20)30535-0. Lancet Oncol. 2021. PMID: 33387492 - PPsNet: An improved deep learning model for microsatellite instability high prediction in colorectal cancer from whole slide images.
Lou J, Xu J, Zhang Y, Sun Y, Fang A, Liu J, Mur LAJ, Ji B. Lou J, et al. Comput Methods Programs Biomed. 2022 Oct;225:107095. doi: 10.1016/j.cmpb.2022.107095. Epub 2022 Aug 28. Comput Methods Programs Biomed. 2022. PMID: 36057226 - Development and validation of MRI-based deep learning models for prediction of microsatellite instability in rectal cancer.
Zhang W, Yin H, Huang Z, Zhao J, Zheng H, He D, Li M, Tan W, Tian S, Song B. Zhang W, et al. Cancer Med. 2021 Jun;10(12):4164-4173. doi: 10.1002/cam4.3957. Epub 2021 May 8. Cancer Med. 2021. PMID: 33963688 Free PMC article. Clinical Trial. - Current Strategy to Treat Immunogenic Gastrointestinal Cancers: Perspectives for a New Era.
Shimozaki K, Nakayama I, Hirota T, Yamaguchi K. Shimozaki K, et al. Cells. 2023 Mar 30;12(7):1049. doi: 10.3390/cells12071049. Cells. 2023. PMID: 37048122 Free PMC article. Review. - State of the art: Targeting microsatellite instability in gastrointestinal cancers.
Mencel J, Alves A, Angelis V, Gerlinger M, Starling N. Mencel J, et al. Crit Rev Oncol Hematol. 2024 Jul;199:104387. doi: 10.1016/j.critrevonc.2024.104387. Epub 2024 May 10. Crit Rev Oncol Hematol. 2024. PMID: 38734279 Review.
Cited by
- Prediction of homologous recombination deficiency from routine histology with attention-based multiple instance learning in nine different tumor types.
Loeffler CML, El Nahhas OSM, Muti HS, Carrero ZI, Seibel T, van Treeck M, Cifci D, Gustav M, Bretz K, Gaisa NT, Lehmann KV, Leary A, Selenica P, Reis-Filho JS, Ortiz-Bruechle N, Kather JN. Loeffler CML, et al. BMC Biol. 2024 Oct 8;22(1):225. doi: 10.1186/s12915-024-02022-9. BMC Biol. 2024. PMID: 39379982 - Tumor Heterogeneity in Gastrointestinal Cancer Based on Multimodal Data Analysis.
Ai D, Du Y, Duan H, Qi J, Wang Y. Ai D, et al. Genes (Basel). 2024 Sep 13;15(9):1207. doi: 10.3390/genes15091207. Genes (Basel). 2024. PMID: 39336798 Free PMC article. - Improving performance in colorectal cancer histology decomposition using deep and ensemble machine learning.
Prezja F, Annala L, Kiiskinen S, Lahtinen S, Ojala T, Ruusuvuori P, Kuopio T. Prezja F, et al. Heliyon. 2024 Sep 10;10(18):e37561. doi: 10.1016/j.heliyon.2024.e37561. eCollection 2024 Sep 30. Heliyon. 2024. PMID: 39309850 Free PMC article. - Precision oncology in patients with breast cancer: towards a 'screen and characterize' approach.
Rassy E, Mosele MF, Di Meglio A, Pistilli B, Andre F. Rassy E, et al. ESMO Open. 2024 Sep 19;9(10):103716. doi: 10.1016/j.esmoop.2024.103716. Online ahead of print. ESMO Open. 2024. PMID: 39303452 Free PMC article. No abstract available. - From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology.
El Nahhas OSM, van Treeck M, Wölflein G, Unger M, Ligero M, Lenz T, Wagner SJ, Hewitt KJ, Khader F, Foersch S, Truhn D, Kather JN. El Nahhas OSM, et al. Nat Protoc. 2024 Sep 16. doi: 10.1038/s41596-024-01047-2. Online ahead of print. Nat Protoc. 2024. PMID: 39285224 Review.
References
- Bonneville R et al. JCO Precis. Oncol. 2017, 1–15 (2017).
- Kather JN, Halama N & Jaeger D Semin. Cancer Biol. 52, 189–197 (2018). - PubMed
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources