A framework to estimate cognitive load using physiological data (original) (raw)
Related papers
Psycho-physiological measures for assessing cognitive load
Proceedings of the 12th ACM international conference on Ubiquitous computing - Ubicomp '10, 2010
With a focus on presenting information at the right time, the ubicomp community can benefit greatly from learning the most salient human measures of cognitive load. Cognitive load can be used as a metric to determine when or whether to interrupt a user. In this paper, we collected data from multiple sensors and compared their ability to assess cognitive load. Our focus is on visual perception and cognitive speed-focused tasks that leverage cognitive abilities common in ubicomp applications. We found that across all participants, the electrocardiogram median absolute deviation and median heat flux measurements were the most accurate at distinguishing between low and high levels of cognitive load, providing a classification accuracy of over 80% when used together. Our contribution is a real-time, objective, and generalizable method for assessing cognitive load in cognitive tasks commonly found in ubicomp systems and situations of divided attention.
Sensitive, Diagnostic and Multifaceted Mental Workload Classifier (PHYSIOPRINT)
Lecture Notes in Computer Science, 2015
Mental workload is difficult to quantify because it results from an interplay of the objective task load, ambient and internal distractions, capacity of mental resources, and strategy of their utilization. Furthermore, different types of mental resources are mobilized to a different degree in different tasks even if their perceived difficulty is the same. Thus, an ideal mental workload measure needs to quantify the degree of utilization of different mental resources in addition to providing a single global workload measure. Here we present a novel assessment tool (called PHYSIOPRINT) that derives workload measures in real time from multiple physiological signals (EEG, ECG, EOG, EMG). PHYSIOPRINT is modeled after the theoretical IMPRINT workload model developed by the US Army that recognizes seven different workload types: auditory, visual, cognitive, speech, tactile, fine motor and gross motor workload. Preliminary investigation on 25 healthy volunteers proved feasibility of the concept and defined the high level system architecture. The classifier was trained on the EEG and ECG data acquired during tasks chosen to represent the key anchors on the respective seven workload scales. The trained model was then validated on realistic driving simulator. The classification accuracy was 88.7% for speech, 86.6% for fine motor, 89.3% for gross motor, 75.8% for auditory, 76.7% for visual, and 72.5% for cognitive workload. By August of 2015, an extended validation of the model will be completed on over 100 volunteers in realistically simulated environments (driving and flight simulator), as well as in a real military-relevant environment (fully instrumented HMMWV).
Cognitive workload level estimation based on eye tracking: A machine learning approach
2021 IEEE 21st International Conference on Bioinformatics and Bioengineering (BIBE), 2021
Cognitive workload is a critical feature in related psychology, ergonomics, and human factors for understanding performance. However, it still is difficult to describe and thus, to measure it. Since there is no single sensor that can give a full understanding of workload, extended research has been conducted in order to present robust biomarkers. During the last years, machine learning techniques have been used to predict cognitive workload based on various features. Gaze extracted features, such as pupil size, blink activity and saccadic measures, have been used as predictors. The aim of this study is to use gaze extracted features as the only predictors of cognitive workload. Two factors were investigated: time pressure and multi tasking. The findings of this study showed that eye and gaze features are useful indicators of cognitive workload levels, reaching up to 88% accuracy.
Towards measuring cognitive load through multimodal physiological data
Cognition, Technology & Work, 2020
Cognitive load plays an important role during learning and working, as it has been linked to wellfunctioning cognitive processes, performance, burnout and depression. Nonetheless, attempts to assess cognitive load in real-time by means of physiological data have been proven difficult, and interpreting these data remains challenging. The aim of this study is to examine whether and how well experienced cognitive load can be measured through psychophysiological data. The approach of this study is rather unique, for a combination of reasons. First, this study takes a multimodal approach, monitoring EDA (electrodermal activity), EEG (electroencephalography) and EOG (electrooculography). Second, this study is based on a relatively intensive data collection (N = 46) in a controlled lab setting in which varying cognitive load levels are deliberately induced. Finally, not only focussing on statistical significance, but also on the size of the association gives insights into how suitable physiological markers are to measure cognitive load. Results from a multilevel analysis suggest that the following physiological markers might be related to cognitive load, for example, in an industrial context: the rate and the duration of skin conductance responses, the alpha power, the alpha peak frequency and the eye blink rate. About 22.8% of the variance in self-reported cognitive load can be explained using these five measures.
Software: Practice and Experience, 2019
Due to the increasing attention to online learning, cognitive load has been recently considered as a crucial indicator for judging teenagers' learning state so as to improve both learning and teaching effects. However, some traditional cognitive load measurement methods such as subjective measurement are easily influenced by subjective sensation deviation of subjects. None of them can reflect the cognitive load of learners more precisely. Recently, machine learning-based data modeling has gained more importance in the scenarios of various smart wearables and Internet of things applications. Meanwhile, physiological signals have proven to contribute much to human health assessment. On the basis of the above considerations, this paper proposes a physiological data-driven model for learners' cognitive load detection under the application of smart wearables. The model consists of four modules: physiological signal acquisition, signal preprocessing, heart rate variability and pulse rate variability feature fusion, and cognitive load classification through an optimized extreme gradient boosting classifier in which hyperparameters are adaptively tuned with sequential model-based optimization. Furthermore, we design an experimental paradigm for signal acquisition in a learning environment, and the experimental results demonstrate that the proposed model for cognitive load detection outperforms conventional approaches that only employ either heart rate variability or pulse rate variability for modeling. We also compare the effects of different feature fusion algorithms combined with different classification algorithms, which demonstrates that the proposed model achieves the highest accuracy of cognitive load detection due to its optimal combination of feature fusion and classification.
Real-Time Prediction of Fluctuations in Cognitive Workload
2020
Human operators often experience large fluctuations in cognitive workload that can lead to sub-optimal performance, ranging from overload to neglect. Help from automated support systems could potentially address this issue, but to do so the system would ideally need to be aware of real-time changes in operators’ cognitive workload, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. We used the ISO standard Detection Response Task (DRT) to measure cognitive workload approximately every 4 seconds in a demanding task requiring monitoring and refuelling of a fleet of unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect changes in workload due to changes in the number of UAVs. We used a cross-validation analysis to assess whether measures related to task performance immediately preceding the DRT could be used to predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators’ situational awareness with respect to fuel levels were much more effective. We conclude that real-time prediction of operators’ cognitive workload shows promise as an avenue for improved human-automation teaming.
Integrated real-time, non-intrusive Measurements for Mental Load
2019
In this position paper, we propose to develop a system that takes input data from different sensors on physiological behaviors such as Pupil Diameter, Blinking Rate, Heart Rate, Heart Rate Variability, and Galvanic Skin Response to estimate users’ mental load (see Sidebar 1). We firstly aim to collect data on these behaviours and then intend to understand the correlation between them and later want to predict an estimate of cognitive load in real-time. We hope to use this measure during Human-Computer Interaction and Human-Robot Interaction to personalize our interfaces or robots’ behaviour according to user mental load in the real-time.
Cognitive Load Estimation in the Wild
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Cognitive load has been shown, over hundreds of validated studies, to be an important variable for understanding human performance. However, establishing practical, non-contact approaches for automated estimation of cognitive load under real-world conditions is far from a solved problem. Toward the goal of designing such a system, we propose two novel vision-based methods for cognitive load estimation, and evaluate them on a large-scale dataset collected under real-world driving conditions. Cognitive load is defined by which of 3 levels of a validated reference task the observed subject was performing. On this 3-class problem, our best proposed method of using 3D convolutional neural networks achieves 86.1% accuracy at predicting task-induced cognitive load in a sample of 92 subjects from video alone. This work uses the driving context as a training and evaluation dataset, but the trained network is not constrained to the driving environment as it requires no calibration and makes no assumptions about the subject's visual appearance, activity, head pose, scale, and perspective.
Cognitive Load Monitoring With Wearables–Lessons Learned From a Machine Learning Challenge
2021
To further extend the applicability of wearable sensors, methods for accurately extracting subtle psychological information from the sensor data are required. However, accessing subjective information in everyday life, such as cognitive load, remains challenging. To bring consensus on methods for cognitive load monitoring, a machine learning challenge is organized. The participants developed machine learning methods for cognitive load classification using wrist-worn physiological sensors’ data, namely heart rate, R-R intervals, skin conductance, and skin temperature. The data from subjects solving cognitive tasks of varying difficulty is used for the challenge. This article presents a systematic comparison and multi-strategic performance evaluation of the thirteen methods submitted to this challenge. A systematic comparison of preprocessing techniques, classification algorithms, and implementation techniques is presented. Performance variations for different task difficulty levels, ...
GSR and Blink Features for Cognitive Load Classification
Human-Computer Interaction – INTERACT 2013, 2013
A system capable of monitoring its user's mental workload can evaluate the suitability of its interface and interactions for user's current cognitive status and properly change them when necessary. Galvanic skin response (GSR) and eye blinks are cognitive load measures which can be captured conveniently and at low cost. The present study has assessed multiple features of these two signals in classification of cognitive workload level. The experiment included arithmetic tasks with four difficulty levels and two types of machine learning algorithms have been applied for classification. Obtained results show that the studied features of blink and GSR can reasonably discriminate workload levels and combining features of the two modalities improves the accuracy of cognitive load classification. We have achieved around 75% for binary classification and more than 50% for four-class classification.