Seyed Mostafa Kia | Radboud University Nijmegen (original) (raw)

Papers by Seyed Mostafa Kia

Research paper thumbnail of Age-related brain deviations and aggression

Psychological Medicine

BackgroundDisruptive behavior disorders (DBD) are heterogeneous at the clinical and the biologica... more BackgroundDisruptive behavior disorders (DBD) are heterogeneous at the clinical and the biological level. Therefore, the aims were to dissect the heterogeneous neurodevelopmental deviations of the affective brain circuitry and provide an integration of these differences across modalities.MethodsWe combined two novel approaches. First, normative modeling to map deviations from the typical age-related pattern at the level of the individual of (i) activity during emotion matching and (ii) of anatomical images derived from DBD cases (n = 77) and controls (n = 52) aged 8–18 years from the EU-funded Aggressotype and MATRICS consortia. Second, linked independent component analysis to integrate subject-specific deviations from both modalities.ResultsWhile cases exhibited on average a higher activity than would be expected for their age during face processing in regions such as the amygdala when compared to controls these positive deviations were widespread at the individual level. A multimo...

Research paper thumbnail of Closing the life-cycle of normative modeling using federated hierarchical Bayesian regression

PLOS ONE, Dec 8, 2022

Clinical neuroimaging data availability has grown substantially in the last decade, providing the... more Clinical neuroimaging data availability has grown substantially in the last decade, providing the potential for studying heterogeneity in clinical cohorts on a previously unprecedented scale. Normative modeling is an emerging statistical tool for dissecting heterogeneity in complex brain disorders. However, its application remains technically challenging due to medical data privacy issues and difficulties in dealing with nuisance variation, such as the variability in the image acquisition process. Here, we approach the problem of estimating a reference normative model across a massive population using a massive multi-center neuroimaging dataset. To this end, we introduce a federated probabilistic framework using hierarchical Bayesian regression (HBR) to complete the life-cycle of normative modeling. The proposed model provides the possibilities to learn, update, and adapt the model parameters on decentralized neuroimaging data. Our experimental results confirm the superiority of HBR in deriving more accurate normative ranges on large multi-site neuroimaging datasets compared to the current standard methods. In addition, our approach provides the possibility to recalibrate and reuse the learned model on local datasets and even on datasets with very small sample sizes. The proposed method will facilitate applications of normative modeling as a medical tool for screening the biological deviations in individuals affected by complex illnesses such as mental disorders.

Research paper thumbnail of Normative Modeling of Neuroimaging Data using Scalable Multi-Task Gaussian Processes

arXiv (Cornell University), Jun 4, 2018

Normative modeling has recently been proposed as an alternative for the case-control approach in ... more Normative modeling has recently been proposed as an alternative for the case-control approach in modeling heterogeneity within clinical cohorts. Normative modeling is based on single-output Gaussian process regression that provides coherent estimates of uncertainty required by the method but does not consider spatial covariance structure. Here, we introduce a scalable multi-task Gaussian process regression (S-MTGPR) approach to address this problem. To this end, we exploit a combination of a low-rank approximation of the spatial covariance matrix with algebraic properties of Kronecker product in order to reduce the computational complexity of Gaussian process regression in high-dimensional output spaces. On a public fMRI dataset, we show that S-MTGPR: 1) leads to substantial computational improvements that allow us to estimate normative models for high-dimensional fMRI data whilst accounting for spatial structure in data; 2) by modeling both spatial and across-sample variances, it provides higher sensitivity in novelty detection scenarios.

Research paper thumbnail of Multi-site Normative Modeling of Diffusion Tensor Imaging Metrics Using Hierarchical Bayesian Regression

Research paper thumbnail of Movie Genre Classification by Exploiting MEG Brain Signals

Springer eBooks, 2015

Genre classification is an essential part of multimedia content recommender systems. In this stud... more Genre classification is an essential part of multimedia content recommender systems. In this study, we provide experimental evidence for the possibility of performing genre classification based on brain recorded signals. The brain decoding paradigm is employed to classify magnetoencephalography (MEG) data presented in [1] to four genre classes: Comedy, Romantic, Drama, and Horror. Our results show that: 1) there is a significant correlation between audiovisual features of movies and corresponding brain signals specially in the visual and temporal lobes; 2) the genre of movie clips can be classified with an accuracy significantly over the chance level using the MEG signal. On top of that we show that the combination of multimedia features and MEG-based features achieves the best accuracy. Our study provides a primary step towards user-centric media content retrieval using brain signals.

Research paper thumbnail of Gender and emotion recognition with implicit user signals

We examine the utility of implicit user behavioral signals captured using low-cost, o-the-shelf d... more We examine the utility of implicit user behavioral signals captured using low-cost, o-the-shelf devices for anonymous gender and emotion recognition. A user study designed to examine male and female sensitivity to facial emotions con rms that females recognize (especially negative) emotions quicker and more accurately than men, mirroring prior ndings. Implicit viewer responses in the form of EEG brain signals and eye movements are then examined for existence of (a) emotion and gender-speci c pa erns from event-related potentials (ERPs) and xation distributions and (b) emotion and gender discriminability. Experiments reveal that (i) Gender and emotion-speci c di erences are observable from ERPs, (ii) multiple similarities exist between explicit responses gathered from users and their implicit behavioral signals, and (iii) Signicantly above-chance (≈70%) gender recognition is achievable on comparing emotion-speci c EEG responses-gender di erences are encoded best for anger and disgust. Also, fairly modest valence (positive vs negative emotion) recognition is achieved with EEG and eye-based features. CCS CONCEPTS •Human-centered computing → HCI theory, concepts and models; User centered design;

Research paper thumbnail of Interpretability of Multivariate Brain Maps in Brain Decoding: Definition and Quantification

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear ... more Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study the spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we formalize a heuristic method for approximating the interpretability of multivariate brain maps in a binary magnetoencephalography (MEG) decoding scenario. Third, we pro

Research paper thumbnail of Gender and Emotion Recognition from Implicit User Behavior Signals

arXiv (Cornell University), Jun 23, 2020

This work explores the utility of implicit behavioral cues, namely, Electroencephalogram (EEG) si... more This work explores the utility of implicit behavioral cues, namely, Electroencephalogram (EEG) signals and eye movements for gender recognition (GR) and emotion recognition (ER) from psychophysical behavior. Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. 28 users (14 male) recognized emotions from unoccluded (no mask) and partially occluded (eye or mouth masked) emotive faces; their EEG responses contained gender-specific differences, while their eye movements were characteristic of the perceived facial emotions. Experimental results reveal that (a) reliable GR and ER is achievable with EEG and eye features, (b) differential cognitive processing of negative emotions is observed for females and (c) eye gaze-based gender differences manifest under partial face occlusion, as typified by the eye and mouth mask conditions.

Research paper thumbnail of Interpretability in Linear Brain Decoding

arXiv (Cornell University), Jun 17, 2016

Improving the interpretability of brain decoding approaches is of primary interest in many neuroi... more Improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of brain decoding models. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, we present a simple definition for interpretability of linear brain decoding models. Then, we propose to combine the interpretability and the performance of the brain decoding into a new multi-objective criterion for model selection. Our preliminary results on the toy data show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative linear models. The presented definition provides the theoretical background for quantitative evaluation of interpretability in linear brain decoding.

Research paper thumbnail of Scalable Multi-Task Gaussian Process Tensor Regression for Normative Modeling of Structured Variation in Neuroimaging Data

arXiv (Cornell University), Jul 31, 2018

Most brain disorders are very heterogeneous in terms of their underlying biology and developing a... more Most brain disorders are very heterogeneous in terms of their underlying biology and developing analysis methods to model such heterogeneity is a major challenge. A promising approach is to use probabilistic regression methods to estimate normative models of brain measures then use these to map variation across individuals. To fully capture individual differences and detect disorders in individual subjects it is crucial to statistically model patterns of correlation across different brain regions and individuals. However, this is very challenging for neuroimaging data because of high dimensionality and highly structured correlations across multiple axes. Here, we propose tensor Gaussian predictive process (TGPP) as a general and flexible Bayesian mixed-effects modeling framework. In TGPP, we develop multi-task Gaussian process tensor regression (MT-GPTR) to simultaneously model the structured random effects and structured noise. We use Kronecker algebra and a low-rank approximation to efficiently scale MT-GPTR to the whole brain. On a publicly available clinical fMRI dataset and in a novelty detection scenario, we show that our computationally affordable multivariate normative modeling approach substantially improves the detection rate over a baseline mass-univariate normative model and an offthe-shelf supervised alternative.

Research paper thumbnail of Non-Gaussian Normative Modelling With Hierarchical Bayesian Regression

bioRxiv (Cold Spring Harbor Laboratory), Oct 7, 2022

• We extended the Hierarchical Bayesian Regression framework for normative modelling • Our extens... more • We extended the Hierarchical Bayesian Regression framework for normative modelling • Our extension allows modelling data with heteroskedastic skewness and kurtosis • We developed a reparameterization of the SHASH distribution, suitable for sampling • We provide the first implementation of the SHASH distribution in a fully Bayesian framework • Results show that the extension outperforms current methods on various measures .

Research paper thumbnail of Stereotypical Motor Movement Detection in Dynamic Feature Space

Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with... more Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with learning and social interaction in Autism Spectrum Disorder patients. An automatic SMM detection system, employing inertial sensing technology, provides a useful tool for real-time alert on the onset of these atypical behaviors, therefore facilitating personalized intervention therapies. To tackle critical issues with inter-subject variability, in this study, we propose to combine long short-term memory (LSTM) with convolutional neural network (CNN) to model the temporal patterns in the sequence of multi-axes IMU signals. Our results, on one simulated and two experimental datasets, show that transferring the raw feature space to a dynamic feature space via the proposed architecture enhances the performance of automatic SMM detection system especially for skewed training data. These findings facilitate the application of SMM detection system in real-time scenarios.

Research paper thumbnail of Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects

Frontiers in Neuroscience, Jan 23, 2017

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear ... more Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we exemplify a heuristic for approximating the interpretability in multivariate analysis of evoked magnetoencephalography (MEG) responses. Third, we propose to combine the approximated interpretability and the generalization performance of the brain decoding into a new multi-objective criterion for model selection. Our results, for the simulated and real MEG data, show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future.

Research paper thumbnail of Hierarchical Bayesian Regression for Multi-site Normative Modeling of Neuroimaging Data

Lecture Notes in Computer Science, 2020

Clinical neuroimaging has recently witnessed explosive growth in data availability which brings s... more Clinical neuroimaging has recently witnessed explosive growth in data availability which brings studying heterogeneity in clinical cohorts to the spotlight. Normative modeling is an emerging statistical tool for achieving this objective. However, its application remains technically challenging due to difficulties in properly dealing with nuisance variation, for example due to variability in image acquisition devices. Here, in a fully probabilistic framework, we propose an application of hierarchical Bayesian regression (HBR) for multi-site normative modeling. Our experimental results confirm the superiority of HBR in deriving more accurate normative ranges on large multi-site neuroimaging data compared to widely used methods. This provides the possibility i) to learn the normative range of structural and functional brain measures on large multi-site data; ii) to recalibrate and reuse the learned model on local small data; therefore, HBR closes the technical loop for applying normative modeling as a medical tool for the diagnosis and prognosis of mental disorders.

Research paper thumbnail of DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses

IEEE Transactions on Affective Computing, Jul 1, 2015

In this work, we present DECAFa multimodal dataset for decoding user physiological responses to a... more In this work, we present DECAFa multimodal dataset for decoding user physiological responses to affective multimedia content. Different from datasets such as DEAP [15] and MAHNOB-HCI [31], DECAF contains (1) brain signals acquired using the Magnetoencephalogram (MEG) sensor, which requires little physical contact with the user's scalp and consequently facilitates naturalistic affective response, and (2) explicit and implicit emotional responses of 30 participants to 40 one-minute music video segments used in [15] and 36 movie clips, thereby enabling comparisons between the EEG vs MEG modalities as well as movie vs music stimuli for affect recognition. In addition to MEG data, DECAF comprises synchronously recorded near-infra-red (NIR) facial videos, horizontal Electrooculogram (hEOG), Electrocardiogram (ECG), and trapezius-Electromyogram (tEMG) peripheral physiological responses. To demonstrate DECAF's utility, we present (i) a detailed analysis of the correlations between participants' self-assessments and their physiological responses and (ii) single-trial classification results for valence, arousal and dominance, with performance evaluation against existing datasets. DECAF also contains time-continuous emotion annotations for movie clips from seven users, which we use to demonstrate dynamic emotion prediction.

Research paper thumbnail of Convolutional Neural Network for Stereotypical Motor Movement Detection in Autism

arXiv (Cornell University), Nov 5, 2015

Autism Spectrum Disorders (ASDs) are associated with specific atypical postural or motor behavior... more Autism Spectrum Disorders (ASDs) are associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) may severely interfere with learning and social interactions. Wireless inertial sensing technology offers a valid infrastructure for automatic and real-time SMM detection, which would provide support for tuned intervention and possibly early alert on the onset of meltdown events. However, the identification and the quantification of SMM patterns remains complex due to strong inter-subject and intra-subject variability, hard to deal with by handcrafted features. Here we propose to employ the deep learning paradigm in order to learn discriminative features directly from multi-sensor accelerometer signals. Our results with convolutional neural networks provide preliminary evidence that feature learning and transfer learning embedded in deep architectures may lead to accurate and robust SMM detectors in longitudinal scenarios.

Research paper thumbnail of Deep Learning for Automatic Stereotypical Motor Movement Detection using Wearable Sensors in Autism Spectrum Disorders

arXiv (Cornell University), Sep 14, 2017

Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor mo... more Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor movements (SMMs) interfere with learning and social interaction. The automatic SMM detection using inertial measurement units (IMU) remains complex due to the strong intra and inter-subject variability, especially when handcrafted features are extracted from the signal. We propose a new application of the deep learning to facilitate automatic SMM detection using multiaxis IMUs. We use a convolutional neural network (CNN) to learn a discriminative feature space from raw data. We show how the CNN can be used for parameter transfer learning to enhance the detection rate on longitudinal data. We also combine the long short-term memory (LSTM) with CNN to model the temporal patterns in a sequence of multi-axis signals. Further, we employ ensemble learning to combine multiple LSTM learners into a more robust SMM detector. Our results show that: 1) feature learning outperforms handcrafted features; 2) parameter transfer learning is beneficial in longitudinal settings; 3) using LSTM to learn the temporal dynamic of signals enhances the detection rate especially for skewed training data; 4) an ensemble of LSTMs provides more accurate and stable detectors. These findings provide a significant step toward accurate SMM detection in real-time scenarios.

Research paper thumbnail of Modeling the cooling performance of vortex tube using a genetic algorithm-based artificial neural network

Thermal Science, 2016

In this study, artificial neural networks have been used to model the effects of four important p... more In this study, artificial neural networks have been used to model the effects of four important parameters consist of the ratio of the length to diameter, the ratio of the cold outlet diameter to the tube diameter, inlet pressure, and cold mass fraction on the cooling performance of counter flow vortex tube. In this approach, experimental data have been used to train and validate the neural network model with MATLAB software. Also, genetic algorithm has been used to find the optimal network architecture. In this model, temperature drop at the cold outlet has been considered as the cooling performance of the vortex tube. Based on experimental data, cooling performance of the vortex tube has been predicted by four inlet parameters. The results of this study indicate that the genetic algorithm-based artificial neural network model is capable of predicting the cooling performance of vortex tube in a wide operating range and with satisfactory precision.

Research paper thumbnail of Accommodating site variation in neuroimaging data using normative and hierarchical Bayesian models

bioRxiv (Cold Spring Harbor Laboratory), Feb 11, 2021

• Development and presentation of normative modeling approach based on hierarchical Bayesian mode... more • Development and presentation of normative modeling approach based on hierarchical Bayesian modeling that can be applied to large multi-site neuroimaging data sets. • Comparison of performance of Hierarchical Bayesian model including site as predictor to several common ways to harmonize for multi-site effects. • Presentation of normative modeling as site correction tool.

Research paper thumbnail of Mapping individualised patterns of atrophy in Alzheimer’s disease using neuroanatomical normative models

Alzheimers & Dementia, Dec 1, 2022

Research paper thumbnail of Age-related brain deviations and aggression

Psychological Medicine

BackgroundDisruptive behavior disorders (DBD) are heterogeneous at the clinical and the biologica... more BackgroundDisruptive behavior disorders (DBD) are heterogeneous at the clinical and the biological level. Therefore, the aims were to dissect the heterogeneous neurodevelopmental deviations of the affective brain circuitry and provide an integration of these differences across modalities.MethodsWe combined two novel approaches. First, normative modeling to map deviations from the typical age-related pattern at the level of the individual of (i) activity during emotion matching and (ii) of anatomical images derived from DBD cases (n = 77) and controls (n = 52) aged 8–18 years from the EU-funded Aggressotype and MATRICS consortia. Second, linked independent component analysis to integrate subject-specific deviations from both modalities.ResultsWhile cases exhibited on average a higher activity than would be expected for their age during face processing in regions such as the amygdala when compared to controls these positive deviations were widespread at the individual level. A multimo...

Research paper thumbnail of Closing the life-cycle of normative modeling using federated hierarchical Bayesian regression

PLOS ONE, Dec 8, 2022

Clinical neuroimaging data availability has grown substantially in the last decade, providing the... more Clinical neuroimaging data availability has grown substantially in the last decade, providing the potential for studying heterogeneity in clinical cohorts on a previously unprecedented scale. Normative modeling is an emerging statistical tool for dissecting heterogeneity in complex brain disorders. However, its application remains technically challenging due to medical data privacy issues and difficulties in dealing with nuisance variation, such as the variability in the image acquisition process. Here, we approach the problem of estimating a reference normative model across a massive population using a massive multi-center neuroimaging dataset. To this end, we introduce a federated probabilistic framework using hierarchical Bayesian regression (HBR) to complete the life-cycle of normative modeling. The proposed model provides the possibilities to learn, update, and adapt the model parameters on decentralized neuroimaging data. Our experimental results confirm the superiority of HBR in deriving more accurate normative ranges on large multi-site neuroimaging datasets compared to the current standard methods. In addition, our approach provides the possibility to recalibrate and reuse the learned model on local datasets and even on datasets with very small sample sizes. The proposed method will facilitate applications of normative modeling as a medical tool for screening the biological deviations in individuals affected by complex illnesses such as mental disorders.

Research paper thumbnail of Normative Modeling of Neuroimaging Data using Scalable Multi-Task Gaussian Processes

arXiv (Cornell University), Jun 4, 2018

Normative modeling has recently been proposed as an alternative for the case-control approach in ... more Normative modeling has recently been proposed as an alternative for the case-control approach in modeling heterogeneity within clinical cohorts. Normative modeling is based on single-output Gaussian process regression that provides coherent estimates of uncertainty required by the method but does not consider spatial covariance structure. Here, we introduce a scalable multi-task Gaussian process regression (S-MTGPR) approach to address this problem. To this end, we exploit a combination of a low-rank approximation of the spatial covariance matrix with algebraic properties of Kronecker product in order to reduce the computational complexity of Gaussian process regression in high-dimensional output spaces. On a public fMRI dataset, we show that S-MTGPR: 1) leads to substantial computational improvements that allow us to estimate normative models for high-dimensional fMRI data whilst accounting for spatial structure in data; 2) by modeling both spatial and across-sample variances, it provides higher sensitivity in novelty detection scenarios.

Research paper thumbnail of Multi-site Normative Modeling of Diffusion Tensor Imaging Metrics Using Hierarchical Bayesian Regression

Research paper thumbnail of Movie Genre Classification by Exploiting MEG Brain Signals

Springer eBooks, 2015

Genre classification is an essential part of multimedia content recommender systems. In this stud... more Genre classification is an essential part of multimedia content recommender systems. In this study, we provide experimental evidence for the possibility of performing genre classification based on brain recorded signals. The brain decoding paradigm is employed to classify magnetoencephalography (MEG) data presented in [1] to four genre classes: Comedy, Romantic, Drama, and Horror. Our results show that: 1) there is a significant correlation between audiovisual features of movies and corresponding brain signals specially in the visual and temporal lobes; 2) the genre of movie clips can be classified with an accuracy significantly over the chance level using the MEG signal. On top of that we show that the combination of multimedia features and MEG-based features achieves the best accuracy. Our study provides a primary step towards user-centric media content retrieval using brain signals.

Research paper thumbnail of Gender and emotion recognition with implicit user signals

We examine the utility of implicit user behavioral signals captured using low-cost, o-the-shelf d... more We examine the utility of implicit user behavioral signals captured using low-cost, o-the-shelf devices for anonymous gender and emotion recognition. A user study designed to examine male and female sensitivity to facial emotions con rms that females recognize (especially negative) emotions quicker and more accurately than men, mirroring prior ndings. Implicit viewer responses in the form of EEG brain signals and eye movements are then examined for existence of (a) emotion and gender-speci c pa erns from event-related potentials (ERPs) and xation distributions and (b) emotion and gender discriminability. Experiments reveal that (i) Gender and emotion-speci c di erences are observable from ERPs, (ii) multiple similarities exist between explicit responses gathered from users and their implicit behavioral signals, and (iii) Signicantly above-chance (≈70%) gender recognition is achievable on comparing emotion-speci c EEG responses-gender di erences are encoded best for anger and disgust. Also, fairly modest valence (positive vs negative emotion) recognition is achieved with EEG and eye-based features. CCS CONCEPTS •Human-centered computing → HCI theory, concepts and models; User centered design;

Research paper thumbnail of Interpretability of Multivariate Brain Maps in Brain Decoding: Definition and Quantification

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear ... more Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study the spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we formalize a heuristic method for approximating the interpretability of multivariate brain maps in a binary magnetoencephalography (MEG) decoding scenario. Third, we pro

Research paper thumbnail of Gender and Emotion Recognition from Implicit User Behavior Signals

arXiv (Cornell University), Jun 23, 2020

This work explores the utility of implicit behavioral cues, namely, Electroencephalogram (EEG) si... more This work explores the utility of implicit behavioral cues, namely, Electroencephalogram (EEG) signals and eye movements for gender recognition (GR) and emotion recognition (ER) from psychophysical behavior. Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. 28 users (14 male) recognized emotions from unoccluded (no mask) and partially occluded (eye or mouth masked) emotive faces; their EEG responses contained gender-specific differences, while their eye movements were characteristic of the perceived facial emotions. Experimental results reveal that (a) reliable GR and ER is achievable with EEG and eye features, (b) differential cognitive processing of negative emotions is observed for females and (c) eye gaze-based gender differences manifest under partial face occlusion, as typified by the eye and mouth mask conditions.

Research paper thumbnail of Interpretability in Linear Brain Decoding

arXiv (Cornell University), Jun 17, 2016

Improving the interpretability of brain decoding approaches is of primary interest in many neuroi... more Improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of brain decoding models. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, we present a simple definition for interpretability of linear brain decoding models. Then, we propose to combine the interpretability and the performance of the brain decoding into a new multi-objective criterion for model selection. Our preliminary results on the toy data show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative linear models. The presented definition provides the theoretical background for quantitative evaluation of interpretability in linear brain decoding.

Research paper thumbnail of Scalable Multi-Task Gaussian Process Tensor Regression for Normative Modeling of Structured Variation in Neuroimaging Data

arXiv (Cornell University), Jul 31, 2018

Most brain disorders are very heterogeneous in terms of their underlying biology and developing a... more Most brain disorders are very heterogeneous in terms of their underlying biology and developing analysis methods to model such heterogeneity is a major challenge. A promising approach is to use probabilistic regression methods to estimate normative models of brain measures then use these to map variation across individuals. To fully capture individual differences and detect disorders in individual subjects it is crucial to statistically model patterns of correlation across different brain regions and individuals. However, this is very challenging for neuroimaging data because of high dimensionality and highly structured correlations across multiple axes. Here, we propose tensor Gaussian predictive process (TGPP) as a general and flexible Bayesian mixed-effects modeling framework. In TGPP, we develop multi-task Gaussian process tensor regression (MT-GPTR) to simultaneously model the structured random effects and structured noise. We use Kronecker algebra and a low-rank approximation to efficiently scale MT-GPTR to the whole brain. On a publicly available clinical fMRI dataset and in a novelty detection scenario, we show that our computationally affordable multivariate normative modeling approach substantially improves the detection rate over a baseline mass-univariate normative model and an offthe-shelf supervised alternative.

Research paper thumbnail of Non-Gaussian Normative Modelling With Hierarchical Bayesian Regression

bioRxiv (Cold Spring Harbor Laboratory), Oct 7, 2022

• We extended the Hierarchical Bayesian Regression framework for normative modelling • Our extens... more • We extended the Hierarchical Bayesian Regression framework for normative modelling • Our extension allows modelling data with heteroskedastic skewness and kurtosis • We developed a reparameterization of the SHASH distribution, suitable for sampling • We provide the first implementation of the SHASH distribution in a fully Bayesian framework • Results show that the extension outperforms current methods on various measures .

Research paper thumbnail of Stereotypical Motor Movement Detection in Dynamic Feature Space

Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with... more Stereotypical Motor Movements (SMMs) are abnormal postural or motor behaviors that interfere with learning and social interaction in Autism Spectrum Disorder patients. An automatic SMM detection system, employing inertial sensing technology, provides a useful tool for real-time alert on the onset of these atypical behaviors, therefore facilitating personalized intervention therapies. To tackle critical issues with inter-subject variability, in this study, we propose to combine long short-term memory (LSTM) with convolutional neural network (CNN) to model the temporal patterns in the sequence of multi-axes IMU signals. Our results, on one simulated and two experimental datasets, show that transferring the raw feature space to a dynamic feature space via the proposed architecture enhances the performance of automatic SMM detection system especially for skewed training data. These findings facilitate the application of SMM detection system in real-time scenarios.

Research paper thumbnail of Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects

Frontiers in Neuroscience, Jan 23, 2017

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear ... more Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we exemplify a heuristic for approximating the interpretability in multivariate analysis of evoked magnetoencephalography (MEG) responses. Third, we propose to combine the approximated interpretability and the generalization performance of the brain decoding into a new multi-objective criterion for model selection. Our results, for the simulated and real MEG data, show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future.

Research paper thumbnail of Hierarchical Bayesian Regression for Multi-site Normative Modeling of Neuroimaging Data

Lecture Notes in Computer Science, 2020

Clinical neuroimaging has recently witnessed explosive growth in data availability which brings s... more Clinical neuroimaging has recently witnessed explosive growth in data availability which brings studying heterogeneity in clinical cohorts to the spotlight. Normative modeling is an emerging statistical tool for achieving this objective. However, its application remains technically challenging due to difficulties in properly dealing with nuisance variation, for example due to variability in image acquisition devices. Here, in a fully probabilistic framework, we propose an application of hierarchical Bayesian regression (HBR) for multi-site normative modeling. Our experimental results confirm the superiority of HBR in deriving more accurate normative ranges on large multi-site neuroimaging data compared to widely used methods. This provides the possibility i) to learn the normative range of structural and functional brain measures on large multi-site data; ii) to recalibrate and reuse the learned model on local small data; therefore, HBR closes the technical loop for applying normative modeling as a medical tool for the diagnosis and prognosis of mental disorders.

Research paper thumbnail of DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses

IEEE Transactions on Affective Computing, Jul 1, 2015

In this work, we present DECAFa multimodal dataset for decoding user physiological responses to a... more In this work, we present DECAFa multimodal dataset for decoding user physiological responses to affective multimedia content. Different from datasets such as DEAP [15] and MAHNOB-HCI [31], DECAF contains (1) brain signals acquired using the Magnetoencephalogram (MEG) sensor, which requires little physical contact with the user's scalp and consequently facilitates naturalistic affective response, and (2) explicit and implicit emotional responses of 30 participants to 40 one-minute music video segments used in [15] and 36 movie clips, thereby enabling comparisons between the EEG vs MEG modalities as well as movie vs music stimuli for affect recognition. In addition to MEG data, DECAF comprises synchronously recorded near-infra-red (NIR) facial videos, horizontal Electrooculogram (hEOG), Electrocardiogram (ECG), and trapezius-Electromyogram (tEMG) peripheral physiological responses. To demonstrate DECAF's utility, we present (i) a detailed analysis of the correlations between participants' self-assessments and their physiological responses and (ii) single-trial classification results for valence, arousal and dominance, with performance evaluation against existing datasets. DECAF also contains time-continuous emotion annotations for movie clips from seven users, which we use to demonstrate dynamic emotion prediction.

Research paper thumbnail of Convolutional Neural Network for Stereotypical Motor Movement Detection in Autism

arXiv (Cornell University), Nov 5, 2015

Autism Spectrum Disorders (ASDs) are associated with specific atypical postural or motor behavior... more Autism Spectrum Disorders (ASDs) are associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) may severely interfere with learning and social interactions. Wireless inertial sensing technology offers a valid infrastructure for automatic and real-time SMM detection, which would provide support for tuned intervention and possibly early alert on the onset of meltdown events. However, the identification and the quantification of SMM patterns remains complex due to strong inter-subject and intra-subject variability, hard to deal with by handcrafted features. Here we propose to employ the deep learning paradigm in order to learn discriminative features directly from multi-sensor accelerometer signals. Our results with convolutional neural networks provide preliminary evidence that feature learning and transfer learning embedded in deep architectures may lead to accurate and robust SMM detectors in longitudinal scenarios.

Research paper thumbnail of Deep Learning for Automatic Stereotypical Motor Movement Detection using Wearable Sensors in Autism Spectrum Disorders

arXiv (Cornell University), Sep 14, 2017

Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor mo... more Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor movements (SMMs) interfere with learning and social interaction. The automatic SMM detection using inertial measurement units (IMU) remains complex due to the strong intra and inter-subject variability, especially when handcrafted features are extracted from the signal. We propose a new application of the deep learning to facilitate automatic SMM detection using multiaxis IMUs. We use a convolutional neural network (CNN) to learn a discriminative feature space from raw data. We show how the CNN can be used for parameter transfer learning to enhance the detection rate on longitudinal data. We also combine the long short-term memory (LSTM) with CNN to model the temporal patterns in a sequence of multi-axis signals. Further, we employ ensemble learning to combine multiple LSTM learners into a more robust SMM detector. Our results show that: 1) feature learning outperforms handcrafted features; 2) parameter transfer learning is beneficial in longitudinal settings; 3) using LSTM to learn the temporal dynamic of signals enhances the detection rate especially for skewed training data; 4) an ensemble of LSTMs provides more accurate and stable detectors. These findings provide a significant step toward accurate SMM detection in real-time scenarios.

Research paper thumbnail of Modeling the cooling performance of vortex tube using a genetic algorithm-based artificial neural network

Thermal Science, 2016

In this study, artificial neural networks have been used to model the effects of four important p... more In this study, artificial neural networks have been used to model the effects of four important parameters consist of the ratio of the length to diameter, the ratio of the cold outlet diameter to the tube diameter, inlet pressure, and cold mass fraction on the cooling performance of counter flow vortex tube. In this approach, experimental data have been used to train and validate the neural network model with MATLAB software. Also, genetic algorithm has been used to find the optimal network architecture. In this model, temperature drop at the cold outlet has been considered as the cooling performance of the vortex tube. Based on experimental data, cooling performance of the vortex tube has been predicted by four inlet parameters. The results of this study indicate that the genetic algorithm-based artificial neural network model is capable of predicting the cooling performance of vortex tube in a wide operating range and with satisfactory precision.

Research paper thumbnail of Accommodating site variation in neuroimaging data using normative and hierarchical Bayesian models

bioRxiv (Cold Spring Harbor Laboratory), Feb 11, 2021

• Development and presentation of normative modeling approach based on hierarchical Bayesian mode... more • Development and presentation of normative modeling approach based on hierarchical Bayesian modeling that can be applied to large multi-site neuroimaging data sets. • Comparison of performance of Hierarchical Bayesian model including site as predictor to several common ways to harmonize for multi-site effects. • Presentation of normative modeling as site correction tool.

Research paper thumbnail of Mapping individualised patterns of atrophy in Alzheimer’s disease using neuroanatomical normative models

Alzheimers & Dementia, Dec 1, 2022