7-Point Checklist and Skin Lesion Classification using Multi-Task Multi-Modal Neural Nets - PubMed (original) (raw)
7-Point Checklist and Skin Lesion Classification using Multi-Task Multi-Modal Neural Nets
Jeremy Kawahara et al. IEEE J Biomed Health Inform. 2018.
Abstract
We propose a multi-task deep convolutional neural network, trained on multi-modal data (clinical and dermoscopic images, and patient meta-data), to classify the 7-point melanoma checklist criteria and perform skin lesion diagnosis. Our neural network is trained using several multi-task loss functions, where each loss considers different combinations of the input modalities, which allows our model to be robust to missing data at inference time. Our final model classifies the 7-point checklist and skin condition diagnosis, produces multi-modal feature vectors suitable for image retrieval, and localizes clinically discriminant regions. We benchmark our approach using 1011 lesion cases, and report comprehensive results over all 7-point criteria and diagnosis. We also make our dataset (images and metadata) publicly available online at http://derm.cs.sfu.ca.
Similar articles
- Skin lesion classification based on two-modal images using a multi-scale fully-shared fusion network.
Yang Y, Xie F, Zhang H, Wang J, Liu J, Zhang Y, Ding H. Yang Y, et al. Comput Methods Programs Biomed. 2023 Feb;229:107315. doi: 10.1016/j.cmpb.2022.107315. Epub 2022 Dec 16. Comput Methods Programs Biomed. 2023. PMID: 36586177 - Learning from dermoscopic images in association with clinical metadata for skin lesion segmentation and classification.
Dong C, Dai D, Zhang Y, Zhang C, Li Z, Xu S. Dong C, et al. Comput Biol Med. 2023 Jan;152:106321. doi: 10.1016/j.compbiomed.2022.106321. Epub 2022 Nov 17. Comput Biol Med. 2023. PMID: 36463792 - A Novel Multi-Task Learning Network Based on Melanoma Segmentation and Classification with Skin Lesion Images.
Alenezi F, Armghan A, Polat K. Alenezi F, et al. Diagnostics (Basel). 2023 Jan 10;13(2):262. doi: 10.3390/diagnostics13020262. Diagnostics (Basel). 2023. PMID: 36673072 Free PMC article. - FusionM4Net: A multi-stage multi-modal learning algorithm for multi-label skin lesion classification.
Tang P, Yan X, Nan Y, Xiang S, Krammer S, Lasser T. Tang P, et al. Med Image Anal. 2022 Feb;76:102307. doi: 10.1016/j.media.2021.102307. Epub 2021 Nov 22. Med Image Anal. 2022. PMID: 34861602 - Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images.
Lihacova I, Bondarenko A, Chizhov Y, Uteshev D, Bliznuks D, Kiss N, Lihachev A. Lihacova I, et al. J Clin Med. 2022 May 17;11(10):2833. doi: 10.3390/jcm11102833. J Clin Med. 2022. PMID: 35628958 Free PMC article.
Cited by
- Skin Type Diversity in Skin Lesion Datasets: A Review.
Alipour N, Burke T, Courtney J. Alipour N, et al. Curr Dermatol Rep. 2024;13(3):198-210. doi: 10.1007/s13671-024-00440-0. Epub 2024 Aug 14. Curr Dermatol Rep. 2024. PMID: 39184010 Free PMC article. Review. - Comprehensive analysis of clinical images contributions for melanoma classification using convolutional neural networks.
Rios-Duarte JA, Diaz-Valencia AC, Combariza G, Feles M, Peña-Silva RA. Rios-Duarte JA, et al. Skin Res Technol. 2024 May;30(5):e13607. doi: 10.1111/srt.13607. Skin Res Technol. 2024. PMID: 38742379 Free PMC article. - Exploring dermoscopic structures for melanoma lesions' classification.
Malik FS, Yousaf MH, Sial HA, Viriri S. Malik FS, et al. Front Big Data. 2024 Mar 25;7:1366312. doi: 10.3389/fdata.2024.1366312. eCollection 2024. Front Big Data. 2024. PMID: 38590699 Free PMC article. - Federated learning for medical image analysis: A survey.
Guan H, Yap PT, Bozoki A, Liu M. Guan H, et al. Pattern Recognit. 2024 Jul;151:110424. doi: 10.1016/j.patcog.2024.110424. Epub 2024 Mar 12. Pattern Recognit. 2024. PMID: 38559674 Free PMC article. - Multi-sample ζ-mixup: richer, more realistic synthetic samples from a _p_-series interpolant.
Abhishek K, Brown CJ, Hamarneh G. Abhishek K, et al. J Big Data. 2024;11(1):43. doi: 10.1186/s40537-024-00898-6. Epub 2024 Mar 23. J Big Data. 2024. PMID: 38528850 Free PMC article.
LinkOut - more resources
Full Text Sources
Other Literature Sources