Nastaran Mohammadian Rad | Maastricht University (original) (raw)
Papers by Nastaran Mohammadian Rad
Frontiers in oncology, May 30, 2024
Objective: Colorectal cancer (CRC) is one of the most prevalent cancers worldwide. A considerable... more Objective: Colorectal cancer (CRC) is one of the most prevalent cancers worldwide. A considerable percentage of patients who undergo surgery with curative intent will experience cancer recurrence. Early identification of individuals with a higher risk of recurrence is crucial for healthcare professionals to intervene promptly and devise appropriate treatment strategies. In this study, we developed prognostic models for CRC recurrence using machine learning models on a limited number of CEA measurements. Method: A dataset of 1927 patients diagnosed with Stage I-III CRC and referred to Zuyderland Hospital for surgery between 2008 and 2016 was utilized. Machine learning models were trained using this comprehensive dataset, which included demographic details, clinicopathological factors, and serial measurements of Carcinoembryonic Antigen (CEA). In this study, the predictive performance of these models was assessed, and the key prognostic factors influencing colorectal cancer (CRC) recurrence were pinpointed Result: Among the evaluated models, the gradient boosting classifier demonstrated superior performance, achieving an Area Under the Curve (AUC) score of 0.81 and a balanced accuracy rate of 0.73. Recurrence prediction was shown to be feasible with an AUC of 0.71 when using only five post-operative CEA measurements. Furthermore, key factors influencing recurrence were identified and elucidated. Conclusion: This study shows the transformative role of machine learning in recurrence prediction for CRC, particularly by investigating the minimum number of CEA measurements required for effective recurrence prediction. This approach not only contributes to the optimization of clinical workflows but also facilitates the development of more effective, individualized treatment plans, thereby laying the groundwork for future advancements in this area. Future directions involve validating these models in larger and more diverse cohorts. Building on these efforts, our ultimate goal is to develop a risk-based follow-up strategy that can improve patient outcomes and enhance healthcare efficiency.
Photoacoustics, Aug 1, 2023
Lecture Notes in Computer Science, 2023
Cancers
Automatic delineation and detection of the primary tumour (GTVp) and lymph nodes (GTVn) using PET... more Automatic delineation and detection of the primary tumour (GTVp) and lymph nodes (GTVn) using PET and CT in head and neck cancer and recurrence-free survival prediction can be useful for diagnosis and patient risk stratification. We used data from nine different centres, with 524 and 359 cases used for training and testing, respectively. We utilised posterior sampling of the weight space in the proposed segmentation model to estimate the uncertainty for false positive reduction. We explored the prognostic potential of radiomics features extracted from the predicted GTVp and GTVn in PET and CT for recurrence-free survival prediction and used SHAP analysis for explainability. We evaluated the bias of models with respect to age, gender, chemotherapy, HPV status, and lesion size. We achieved an aggregate Dice score of 0.774 and 0.760 on the test set for GTVp and GTVn, respectively. We observed a per image false positive reduction of 19.5% and 7.14% using the uncertainty threshold for GT...
arXiv (Cornell University), Jun 3, 2022
While data are the primary fuel for machine learning models, they often suffer from missing value... more While data are the primary fuel for machine learning models, they often suffer from missing values, especially when collected in real-world scenarios. However, many off-the-shelf machine learning models, including artificial neural network models, are unable to handle these missing values directly. Therefore, extra data preprocessing and curation steps, such as data imputation, are inevitable before learning and prediction processes. In this study, we propose a simple and intuitive yet effective method for pruning missing values (PROMISSING) during learning and inference steps in neural networks. In this method, there is no need to remove or impute the missing values; instead, the missing values are treated as a new source of information (representing what we do not know). Our experiments on simulated data, several classification and regression benchmarks, and a multi-modal clinical dataset show that PROMISSING results in similar prediction performance compared to various imputation techniques. In addition, our experiments show models trained using PROMISSING techniques are becoming less decisive in their predictions when facing incomplete samples with many unknowns. This finding hopefully advances machine learning models from being pure predicting machines to more realistic thinkers that can also say "I do not know" when facing incomplete sources of information.
Biomedicines
(1) Background: The main aim was to develop a prototype application that would serve as an open-s... more (1) Background: The main aim was to develop a prototype application that would serve as an open-source repository for a curated subset of predictive and prognostic models regarding oncology, and provide a user-friendly interface for the included models to allow online calculation. The focus of the application is on providing physicians and health professionals with patient-specific information regarding treatment plans, survival rates, and side effects for different expected treatments. (2) Methods: The primarily used models were the ones developed by our research group in the past. This selection was completed by a number of models, addressing the same cancer types but focusing on other outcomes that were selected based on a literature search in PubMed and Medline databases. All selected models were publicly available and had been validated TRIPOD (Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis) type 3 or 2b. (3) Results: The open source repository currently incorporates 18 models from different research groups, evaluated on datasets from different countries. Model types included logistic regression, Cox regression, and recursive partition analysis (decision trees). (4) Conclusions: An application was developed to enable physicians to complement their clinical judgment with user-friendly patient-specific predictions using models that have received internal/external validation. Additionally, this platform enables researchers to display their work, enhancing the use and exposure of their models.
Signal Processing
Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor mo... more Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor movements (SMMs) interfere with learning and social interaction. The automatic SMM detection using inertial measurement units (IMU) remains complex due to the strong intra and inter-subject variability, especially when handcrafted features are extracted from the signal. We propose a new application of the deep learning to facilitate automatic SMM detection using multiaxis IMUs. We use a convolutional neural network (CNN) to learn a discriminative feature space from raw data. We show how the CNN can be used for parameter transfer learning to enhance the detection rate on longitudinal data. We also combine the long short-term memory (LSTM) with CNN to model the temporal patterns in a sequence of multi-axis signals. Further, we employ ensemble learning to combine multiple LSTM learners into a more robust SMM detector. Our results show that: 1) feature learning outperforms handcrafted features; 2) parameter transfer learning is beneficial in longitudinal settings; 3) using LSTM to learn the temporal dynamic of signals enhances the detection rate especially for skewed training data; 4) an ensemble of LSTMs provides more accurate and stable detectors. These findings provide a significant step toward accurate SMM detection in real-time scenarios.
2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), 2016
Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with... more Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with learning and social interaction in Autism Spectrum Disorder patients. An automatic SMM detection system, employing inertial sensing technology, provides a useful tool for real-time alert on the onset of these atypical behaviors, therefore facilitating personalized intervention therapies. To tackle critical issues with inter-subject variability, in this study, we propose to combine long short-term memory (LSTM) with convolutional neural network (CNN) to model the temporal patterns in the sequence of multi-axes IMU signals. Our results, on one simulated and two experimental datasets, show that transferring the raw feature space to a dynamic feature space via the proposed architecture enhances the performance of automatic SMM detection system especially for skewed training data. These findings facilitate the application of SMM detection system in real-time scenarios.
2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), 2016
Autism Spectrum Disorders (ASD) are often associated with specific atypical postural or motor beh... more Autism Spectrum Disorders (ASD) are often associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) interfere with learning and social interaction. Wireless inertial sensing technology offers a valid infrastructure for real-time SMM detection, whose automation would provide support for tuned intervention and possibly early alert on the onset of meltdown events. The identification and the quantification of SMM patterns remains complex due to strong inter-subject and intra-subject variability, in particular when handcrafted features are considered. This study aims at developing automatic SMM detection systems in a real world setting, based on a deep learning architecture. Here, after a review of the current state of the art of automatic SMM detection, we propose to employ the deep learning paradigm in order to learn the discriminating features from multi-sensor accelerometer signals. Our results with convolutional neural networks provided the preliminary evidence that feature learning and transfer learning embedded in deep architectures can provide accurate SMM detectors in longitudinal scenarios.
Sensors
Detecting and monitoring of abnormal movement behaviors in patients with Parkinson’s Disease (PD)... more Detecting and monitoring of abnormal movement behaviors in patients with Parkinson’s Disease (PD) and individuals with Autism Spectrum Disorders (ASD) are beneficial for adjusting care and medical treatment in order to improve the patient’s quality of life. Supervised methods commonly used in the literature need annotation of data, which is a time-consuming and costly process. In this paper, we propose deep normative modeling as a probabilistic novelty detection method, in which we model the distribution of normal human movements recorded by wearable sensors and try to detect abnormal movements in patients with PD and ASD in a novelty detection framework. In the proposed deep normative model, a movement disorder behavior is treated as an extreme of the normal range or, equivalently, as a deviation from the normal movements. Our experiments on three benchmark datasets indicate the effectiveness of the proposed method, which outperforms one-class SVM and the reconstruction-based novel...
ArXiv, 2015
Autism Spectrum Disorders (ASDs) are often associated with specific atypical postural or motor be... more Autism Spectrum Disorders (ASDs) are often associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) have a specific visibility. While the identification and the quantification of SMM patterns remain complex, its automation would provide support to accurate tuning of the intervention in the therapy of autism. Therefore, it is essential to develop automatic SMM detection systems in a real world setting, taking care of strong inter-subject and intra-subject variability. Wireless accelerometer sensing technology can provide a valid infrastructure for real-time SMM detection, however such variability remains a problem also for machine learning methods, in particular whenever handcrafted features extracted from accelerometer signal are considered. Here, we propose to employ the deep learning paradigm in order to learn discriminating features from multi-sensor accelerometer signals. Our results provide preliminary evidence that feature le...
ArXiv, 2021
Ultrasound-guided regional anesthesia (UGRA) can replace general anesthesia (GA), improving pain ... more Ultrasound-guided regional anesthesia (UGRA) can replace general anesthesia (GA), improving pain control and recovery time. This method can be applied on the brachial plexus (BP) after clavicular surgeries. However, identification of the BP from ultrasound (US) images is difficult, even for trained professionals. To address this problem, convolutional neural networks (CNNs) and more advanced deep neural networks (DNNs) can be used for identification and segmentation of the BP nerve region. In this paper, we propose a hybrid model consisting of a classification model followed by a segmentation model to segment BP nerve regions in ultrasound images. A CNN model is employed as a classifier to precisely select the images with the BP region. Then, a U-net or M-net model is used for the segmentation. Our experimental results indicate that the proposed hybrid model significantly improves the segmentation performance over a single segmentation model.
Sensors
Cardiovascular diseases (CVDs) remain the leading cause of death worldwide. An effective manageme... more Cardiovascular diseases (CVDs) remain the leading cause of death worldwide. An effective management and treatment of CVDs highly relies on accurate diagnosis of the disease. As the most common imaging technique for clinical diagnosis of the CVDs, US imaging has been intensively explored. Especially with the introduction of deep learning (DL) techniques, US imaging has advanced tremendously in recent years. Photoacoustic imaging (PAI) is one of the most promising new imaging methods in addition to the existing clinical imaging methods. It can characterize different tissue compositions based on optical absorption contrast and thus can assess the functionality of the tissue. This paper reviews some major technological developments in both US (combined with deep learning techniques) and PA imaging in the application of diagnosis of CVDs.
npj Digital Medicine
Consumer wearables and sensors are a rich source of data about patients’ daily disease and sympto... more Consumer wearables and sensors are a rich source of data about patients’ daily disease and symptom burden, particularly in the case of movement disorders like Parkinson’s disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).
Mobile health, the collection of data using wearables and sensors, is a rapidly growing field in ... more Mobile health, the collection of data using wearables and sensors, is a rapidly growing field in health research with many applications. Deriving validated measures of disease and severity that can be used clinically or as outcome measures in clinical trials, referred to as digital biomarkers, has proven difficult. In part due to the complicated analytical approaches necessary to develop these metrics. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of Parkinson’s Disease (PD) and severity of three PD symptoms: tremor, dyskinesia and bradykinesia. 40 teams from around the world submitted features, and achieved drastically improved predictive performance for PD (best AUROC=0.87), as well as severity of tremor (best AUPR=0.75), dyskinesia (best AUPR=0.48) and bradykinesia (best AUPR=0.95).
Frontiers in oncology, May 30, 2024
Objective: Colorectal cancer (CRC) is one of the most prevalent cancers worldwide. A considerable... more Objective: Colorectal cancer (CRC) is one of the most prevalent cancers worldwide. A considerable percentage of patients who undergo surgery with curative intent will experience cancer recurrence. Early identification of individuals with a higher risk of recurrence is crucial for healthcare professionals to intervene promptly and devise appropriate treatment strategies. In this study, we developed prognostic models for CRC recurrence using machine learning models on a limited number of CEA measurements. Method: A dataset of 1927 patients diagnosed with Stage I-III CRC and referred to Zuyderland Hospital for surgery between 2008 and 2016 was utilized. Machine learning models were trained using this comprehensive dataset, which included demographic details, clinicopathological factors, and serial measurements of Carcinoembryonic Antigen (CEA). In this study, the predictive performance of these models was assessed, and the key prognostic factors influencing colorectal cancer (CRC) recurrence were pinpointed Result: Among the evaluated models, the gradient boosting classifier demonstrated superior performance, achieving an Area Under the Curve (AUC) score of 0.81 and a balanced accuracy rate of 0.73. Recurrence prediction was shown to be feasible with an AUC of 0.71 when using only five post-operative CEA measurements. Furthermore, key factors influencing recurrence were identified and elucidated. Conclusion: This study shows the transformative role of machine learning in recurrence prediction for CRC, particularly by investigating the minimum number of CEA measurements required for effective recurrence prediction. This approach not only contributes to the optimization of clinical workflows but also facilitates the development of more effective, individualized treatment plans, thereby laying the groundwork for future advancements in this area. Future directions involve validating these models in larger and more diverse cohorts. Building on these efforts, our ultimate goal is to develop a risk-based follow-up strategy that can improve patient outcomes and enhance healthcare efficiency.
Photoacoustics, Aug 1, 2023
Lecture Notes in Computer Science, 2023
Cancers
Automatic delineation and detection of the primary tumour (GTVp) and lymph nodes (GTVn) using PET... more Automatic delineation and detection of the primary tumour (GTVp) and lymph nodes (GTVn) using PET and CT in head and neck cancer and recurrence-free survival prediction can be useful for diagnosis and patient risk stratification. We used data from nine different centres, with 524 and 359 cases used for training and testing, respectively. We utilised posterior sampling of the weight space in the proposed segmentation model to estimate the uncertainty for false positive reduction. We explored the prognostic potential of radiomics features extracted from the predicted GTVp and GTVn in PET and CT for recurrence-free survival prediction and used SHAP analysis for explainability. We evaluated the bias of models with respect to age, gender, chemotherapy, HPV status, and lesion size. We achieved an aggregate Dice score of 0.774 and 0.760 on the test set for GTVp and GTVn, respectively. We observed a per image false positive reduction of 19.5% and 7.14% using the uncertainty threshold for GT...
arXiv (Cornell University), Jun 3, 2022
While data are the primary fuel for machine learning models, they often suffer from missing value... more While data are the primary fuel for machine learning models, they often suffer from missing values, especially when collected in real-world scenarios. However, many off-the-shelf machine learning models, including artificial neural network models, are unable to handle these missing values directly. Therefore, extra data preprocessing and curation steps, such as data imputation, are inevitable before learning and prediction processes. In this study, we propose a simple and intuitive yet effective method for pruning missing values (PROMISSING) during learning and inference steps in neural networks. In this method, there is no need to remove or impute the missing values; instead, the missing values are treated as a new source of information (representing what we do not know). Our experiments on simulated data, several classification and regression benchmarks, and a multi-modal clinical dataset show that PROMISSING results in similar prediction performance compared to various imputation techniques. In addition, our experiments show models trained using PROMISSING techniques are becoming less decisive in their predictions when facing incomplete samples with many unknowns. This finding hopefully advances machine learning models from being pure predicting machines to more realistic thinkers that can also say "I do not know" when facing incomplete sources of information.
Biomedicines
(1) Background: The main aim was to develop a prototype application that would serve as an open-s... more (1) Background: The main aim was to develop a prototype application that would serve as an open-source repository for a curated subset of predictive and prognostic models regarding oncology, and provide a user-friendly interface for the included models to allow online calculation. The focus of the application is on providing physicians and health professionals with patient-specific information regarding treatment plans, survival rates, and side effects for different expected treatments. (2) Methods: The primarily used models were the ones developed by our research group in the past. This selection was completed by a number of models, addressing the same cancer types but focusing on other outcomes that were selected based on a literature search in PubMed and Medline databases. All selected models were publicly available and had been validated TRIPOD (Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis) type 3 or 2b. (3) Results: The open source repository currently incorporates 18 models from different research groups, evaluated on datasets from different countries. Model types included logistic regression, Cox regression, and recursive partition analysis (decision trees). (4) Conclusions: An application was developed to enable physicians to complement their clinical judgment with user-friendly patient-specific predictions using models that have received internal/external validation. Additionally, this platform enables researchers to display their work, enhancing the use and exposure of their models.
Signal Processing
Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor mo... more Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor movements (SMMs) interfere with learning and social interaction. The automatic SMM detection using inertial measurement units (IMU) remains complex due to the strong intra and inter-subject variability, especially when handcrafted features are extracted from the signal. We propose a new application of the deep learning to facilitate automatic SMM detection using multiaxis IMUs. We use a convolutional neural network (CNN) to learn a discriminative feature space from raw data. We show how the CNN can be used for parameter transfer learning to enhance the detection rate on longitudinal data. We also combine the long short-term memory (LSTM) with CNN to model the temporal patterns in a sequence of multi-axis signals. Further, we employ ensemble learning to combine multiple LSTM learners into a more robust SMM detector. Our results show that: 1) feature learning outperforms handcrafted features; 2) parameter transfer learning is beneficial in longitudinal settings; 3) using LSTM to learn the temporal dynamic of signals enhances the detection rate especially for skewed training data; 4) an ensemble of LSTMs provides more accurate and stable detectors. These findings provide a significant step toward accurate SMM detection in real-time scenarios.
2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), 2016
Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with... more Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with learning and social interaction in Autism Spectrum Disorder patients. An automatic SMM detection system, employing inertial sensing technology, provides a useful tool for real-time alert on the onset of these atypical behaviors, therefore facilitating personalized intervention therapies. To tackle critical issues with inter-subject variability, in this study, we propose to combine long short-term memory (LSTM) with convolutional neural network (CNN) to model the temporal patterns in the sequence of multi-axes IMU signals. Our results, on one simulated and two experimental datasets, show that transferring the raw feature space to a dynamic feature space via the proposed architecture enhances the performance of automatic SMM detection system especially for skewed training data. These findings facilitate the application of SMM detection system in real-time scenarios.
2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), 2016
Autism Spectrum Disorders (ASD) are often associated with specific atypical postural or motor beh... more Autism Spectrum Disorders (ASD) are often associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) interfere with learning and social interaction. Wireless inertial sensing technology offers a valid infrastructure for real-time SMM detection, whose automation would provide support for tuned intervention and possibly early alert on the onset of meltdown events. The identification and the quantification of SMM patterns remains complex due to strong inter-subject and intra-subject variability, in particular when handcrafted features are considered. This study aims at developing automatic SMM detection systems in a real world setting, based on a deep learning architecture. Here, after a review of the current state of the art of automatic SMM detection, we propose to employ the deep learning paradigm in order to learn the discriminating features from multi-sensor accelerometer signals. Our results with convolutional neural networks provided the preliminary evidence that feature learning and transfer learning embedded in deep architectures can provide accurate SMM detectors in longitudinal scenarios.
Sensors
Detecting and monitoring of abnormal movement behaviors in patients with Parkinson’s Disease (PD)... more Detecting and monitoring of abnormal movement behaviors in patients with Parkinson’s Disease (PD) and individuals with Autism Spectrum Disorders (ASD) are beneficial for adjusting care and medical treatment in order to improve the patient’s quality of life. Supervised methods commonly used in the literature need annotation of data, which is a time-consuming and costly process. In this paper, we propose deep normative modeling as a probabilistic novelty detection method, in which we model the distribution of normal human movements recorded by wearable sensors and try to detect abnormal movements in patients with PD and ASD in a novelty detection framework. In the proposed deep normative model, a movement disorder behavior is treated as an extreme of the normal range or, equivalently, as a deviation from the normal movements. Our experiments on three benchmark datasets indicate the effectiveness of the proposed method, which outperforms one-class SVM and the reconstruction-based novel...
ArXiv, 2015
Autism Spectrum Disorders (ASDs) are often associated with specific atypical postural or motor be... more Autism Spectrum Disorders (ASDs) are often associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) have a specific visibility. While the identification and the quantification of SMM patterns remain complex, its automation would provide support to accurate tuning of the intervention in the therapy of autism. Therefore, it is essential to develop automatic SMM detection systems in a real world setting, taking care of strong inter-subject and intra-subject variability. Wireless accelerometer sensing technology can provide a valid infrastructure for real-time SMM detection, however such variability remains a problem also for machine learning methods, in particular whenever handcrafted features extracted from accelerometer signal are considered. Here, we propose to employ the deep learning paradigm in order to learn discriminating features from multi-sensor accelerometer signals. Our results provide preliminary evidence that feature le...
ArXiv, 2021
Ultrasound-guided regional anesthesia (UGRA) can replace general anesthesia (GA), improving pain ... more Ultrasound-guided regional anesthesia (UGRA) can replace general anesthesia (GA), improving pain control and recovery time. This method can be applied on the brachial plexus (BP) after clavicular surgeries. However, identification of the BP from ultrasound (US) images is difficult, even for trained professionals. To address this problem, convolutional neural networks (CNNs) and more advanced deep neural networks (DNNs) can be used for identification and segmentation of the BP nerve region. In this paper, we propose a hybrid model consisting of a classification model followed by a segmentation model to segment BP nerve regions in ultrasound images. A CNN model is employed as a classifier to precisely select the images with the BP region. Then, a U-net or M-net model is used for the segmentation. Our experimental results indicate that the proposed hybrid model significantly improves the segmentation performance over a single segmentation model.
Sensors
Cardiovascular diseases (CVDs) remain the leading cause of death worldwide. An effective manageme... more Cardiovascular diseases (CVDs) remain the leading cause of death worldwide. An effective management and treatment of CVDs highly relies on accurate diagnosis of the disease. As the most common imaging technique for clinical diagnosis of the CVDs, US imaging has been intensively explored. Especially with the introduction of deep learning (DL) techniques, US imaging has advanced tremendously in recent years. Photoacoustic imaging (PAI) is one of the most promising new imaging methods in addition to the existing clinical imaging methods. It can characterize different tissue compositions based on optical absorption contrast and thus can assess the functionality of the tissue. This paper reviews some major technological developments in both US (combined with deep learning techniques) and PA imaging in the application of diagnosis of CVDs.
npj Digital Medicine
Consumer wearables and sensors are a rich source of data about patients’ daily disease and sympto... more Consumer wearables and sensors are a rich source of data about patients’ daily disease and symptom burden, particularly in the case of movement disorders like Parkinson’s disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).
Mobile health, the collection of data using wearables and sensors, is a rapidly growing field in ... more Mobile health, the collection of data using wearables and sensors, is a rapidly growing field in health research with many applications. Deriving validated measures of disease and severity that can be used clinically or as outcome measures in clinical trials, referred to as digital biomarkers, has proven difficult. In part due to the complicated analytical approaches necessary to develop these metrics. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of Parkinson’s Disease (PD) and severity of three PD symptoms: tremor, dyskinesia and bradykinesia. 40 teams from around the world submitted features, and achieved drastically improved predictive performance for PD (best AUROC=0.87), as well as severity of tremor (best AUPR=0.75), dyskinesia (best AUPR=0.48) and bradykinesia (best AUPR=0.95).