Estimating mental workload through event-related fluctuations of pupil area during a task in a virtual world (original) (raw)
Related papers
Cognitive load estimation in VR flight simulator
Journal of Eye Movement Research
This paper discusses the design and development of a low-cost virtual reality (VR) based flight simulator with cognitive load estimation feature using ocular and EEG signals. Focus is on exploring methods to evaluate pilot’s interactions with aircraft by means of quantifying pilot’s perceived cognitive load under different task scenarios. Realistic target tracking and context of the battlefield is designed in VR. Head mounted eye gaze tracker and EEG headset are used for acquiring pupil diameter, gaze fixation, gaze direction and EEG theta, alpha, and beta band power data in real time. We developed an AI agent model in VR and created scenarios of interactions with the piloted aircraft. To estimate the pilot’s cognitive load, we used low-frequency pupil diameter variations, fixation rate, gaze distribution pattern, EEG signal-based task load index and EEG task engagement index. We compared the physiological measures of workload with the standard user’s inceptor control-based workloa...
Task-evoked pupillary response to mental workload in human-computer interaction
Extended abstracts of the 2004 conference on Human factors and computing systems - CHI '04, 2004
Accurate assessment of a user's mental workload will be critical for developing systems that manage user attention (interruptions) in the user interface. Empirical evidence suggests that an interruption is much less disruptive when it occurs during a period of lower mental workload. To provide a measure of mental workload for interactive tasks, we investigated the use of task-evoked pupillary response. Results show that a more difficult task demands longer processing time, induces higher subjective ratings of mental workload, and reliably evokes greater pupillary response at salient subtasks. We discuss the findings and their implications for the design of an attention manager.
Concurrent validity of an ocular measure of mental workload
In previous studies, eye fixations were recorded from participants playing a videogame and from professional pilots during a simulated flight. Ocular data were then analyzed using spatial statistics algorithms, and results showed sensitivity of fixations' dispersion to variations in mental workload. Particularly, a tendency towards spatial randomness of fixations was associated to the most demanding phases. Implementation of this procedure is still in its early stage, thus making it necessary to assess its validity. In the present study, the index has been used to assess mental workload during the execution of another visuo-motor task: the Tetris game. Task demand was manipulated by varying the degree of difficulty of the game. This was accomplished implementing three levels of difficulty of the game that were selected in a pilot study involving a sample of gamers. Additionally, the amplitude of the P300 component of ERPs was used as a concurrent measure of mental workload. Results showed that fixations dispersion is a valid index of mental workload.
Multi-Modal Measurements of Mental Load
ArXiv, 2019
This position paper describes an experiment conducted to understand the relationships between different physiological measures including pupil Diameter, Blinking Rate, Heart Rate, and Heart Rate Variability in order to develop an estimation of users' mental load in real-time (see Sidebar 1). Our experiment involved performing a task to spot a correct or an incorrect word or sentence with different difficulties in order to induce mental load. We briefly present the analysis of task performance and response time for the items of the experiment task.
Frontiers in neuroscience, 2014
While studies exist that compare different physiological variables with respect to their association with mental workload, it is still largely unclear which variables supply the best information about momentary workload of an individual and what is the benefit of combining them. We investigated workload using the n-back task, controlling for body movements and visual input. We recorded EEG, skin conductance, respiration, ECG, pupil size and eye blinks of 14 subjects. Various variables were extracted from these recordings and used as features in individually tuned classification models. Online classification was simulated by using the first part of the data as training set and the last part of the data for testing the models. The results indicate that EEG performs best, followed by eye related measures and peripheral physiology. Combining variables from different sensors did not significantly improve workload assessment over the best performing sensor alone. Best classification accur...
Eye Tracking-Based Workload and Performance Assessment for Skill Acquisition
Advances in Neuroergonomics and Cognitive Engineering, 2019
The result of training to improve in a given skill is most often demonstrated by an increase in the relevant performance measures. However, a complementary and at times more informative measure is the mental workload imposed on the performer when doing the task. While a number of varied methods exist for measuring workload, we have chosen to explore physiological and neurological correlates for their low amount of impact and interference on subjects during an experiment. In this study, participants trained on a six-task cognitive battery over four weeks while being simultaneously recorded with remote eye tracking and a host of other neurophysiological instruments. In this preliminary analysis, we found that measures of saccades, fixations, and pupil diameters significantly correlated with task performance over time and at different difficulties, indicating the validity of our task battery as well as the specificity of workload-related eye tracking measures.
Journal of Computer Assisted Learning, 2023
Background Cognitive load (CL) management is essential in safety‐critical fields so that professionals can monitor and control their cognitive resources efficiently to perform and solve scenarios in a timely and safe manner, even in complex and unexpected circumstances. Thus, cognitive load theory (CLT) can be used to design virtual reality (VR) training programmes for professional learning in these fields. Objectives We studied CL management performance through behavioural indicators in authentic VR flight training and explored if and to what extent physiological data was associated with CL management performance. Methods The expert (n = 8) and novice pilots (n = 6) performed three approach and landing scenarios with increasing element interactivity. We used video recordings of the training to assess CL management performance based on the behavioural indicators. Then, we used the heart rate (HR) and heart rate variability (HRV) data to study the associations between the physiological data and CL management performance. Results and Conclusions The pilots performed effectively in CL management. The experience of the pilots did not remarkably explain the variation in CL management performance. The scenario with the highest element interactivity and an increase in the very low‐frequency band of HRV were associated with decreased performance in CL management. Takeaways Our study sheds light on the association between physiological indicators and CL management performance, which has traditionally been assessed with behavioural indicators in professional learning in safety‐critical fields. Thus, physiological measurements can be used to supplement the assessment of CL management performance, as relying solely on behavioural indicators can be time consuming.
Evaluating EEG Measures as a Workload Assessment in an Operational Video Game Setup
We tested the electroencephalography (EEG) B-Alert X10 system (Advance Brain Monitoring, Inc.) mental workload metrics. When we evaluate a human-systems interfaces (HSI), we need to assess the operator's state during a task in order evaluate the systems efficiency at helping the operator. Physiological metrics are of good help when it comes to evaluate the operator's mental workload, and EEG is a promising tool. The B-Alert system includes an internal signal processing algorithm computing a mental workload index. We set up a simple experiment on a video game in order to evaluate the reliability of this index. Participants were asked to play a video game with different levels of goal (easy vs hard) as we measured subjective, behavioral and physiological indices (B-Alert mental workload index, pupillometry) of mental workload.
Evaluation of Subjective and EEG-Based Measures of Mental Workload
Communications in Computer and Information Science, 2013
Assessment of mental workload is an important aspect of many human factors and HCI applications. Not surprisingly, a number of workload measures have been proposed. This study examined the sensitivity, convergent and concurrent validity of several subjective self-report and EEG workload measures. Most measures displayed adequate sensitivity to task difficulty manipulations, but relatively modest convergent and concurrent validity. Overall, we believe these result serve to aid human factors practitioners in selecting measures of workload for varied applications.
Applying Real Time Physiological Measures of Cognitive Load to Improve Training
Lecture Notes in Computer Science, 2009
This paper discusses how the fields of augmented cognition and neuroergonomics can be expanded into training. Several classification algorithms based upon EEG data and occular data are discussed in terms of their ability to classify operator state in real time. These indices have been shown to enhance operator performance within adaptive automation paradigms. Learning is different from performing a task that one is familiar with. According to cognitive load theory (CLT), learning is essentially the act of organizing information from working memory into long term memory. However, our working memory system has a bottleneck in this process, such that when training exceeds working memory capacity, learning is hindered. This paper discusses how CLT can be combined with multiple resource theory to create a model of adaptive training. This new paradigm hypothesizes that a system that can monitor working memory capacity in real time and adjust training difficulty can improve learning.