The Role of Personalization and Multiple EEG and Sound Features Selection in Real Time Sonification for Neurofeedback (original) (raw)

Real-Time Electroencephalogram Sonification for Neurofeedback

2018

Electroencephalography (EEG) is the measurement via the scalp of the electrical activity of the brain. The established therapeutic intervention of neurofeedback involves presenting people with their own EEG in real-time to enable them to modify their EEG for purposes of improving performance or health. The aim of this research is to develop and validate real-time sonifications of EEG for use in neurofeedback and methods for assessing such sonifications. Neurofeedback generally uses a visual display. Where auditory feedback is used, it is mostly limited to pre-recorded sounds triggered by the EEG activity crossing a threshold. However, EEG generates time-series data with meaningful detail at fine temporal resolution and with complex temporal dynamics. Human hearing has a much higher temporal resolution than human vision, and auditory displays do not require people to focus on a screen with their eyes open for extended periods of time – e.g. if they are engaged in some other task. Son...

Prototyping a Method for the Assessment of Real-Time Eeg Sonifications

This paper presents a first step in the development of a methodology to compare the ability of different sonifications to convey the fine temporal detail of the Electroencephalography (EEG) brainwave signal in real time. In EEG neurofeedback a person " s EEG activity is monitored and presented back to them, to help them to learn how to modify their brain activity. Learning theory suggests that the more rapidly and accurately the feedback follows behaviour the more efficient the learning will be. Therefore a critical issue is how to assess the ability of a sonification to convey rapid and temporally complex EEG data for neurofeedback. To allow for replication, this study used sonifications of pre-recorded EEG data and asked participants to try and track aspects of the signal in real time using a mouse. This study showed that, although imperfect, this approach is a practical way to compare the suitability of EEG sonifications for tracking detailed EEG signals in real time and tha...

A review of real-time EEG sonification research

Over the last few decades there has been steady growth in research that addresses the real-time sonification of electroencephalographic (EEG) data. Diverse application areas include medical data screening, Brain Computer Interfaces (BCI), neurofeedback, affective computing and applications in the arts. The present paper presents an overview and critical review of the principal research to date in EEG data sonification. Firstly, we identify several sub-domains of real-time EEG sonification and discuss their diverse approaches and goals. Secondly, we describe our search and inclusion criteria, and then present a synoptic summary table spanning over fifty different research projects or published research findings. Thirdly, we analyze sonification approaches to the various EEG data dimensions such as time-frequency filtering, signal level, location, before going on to consider higher order EEG features. Finally, we discuss future application domains which may benefit from new capabilities in the real-time sonification of EEG data. We believe that the present critical review may help to reduce research fragmentation and may aid future collaboration in this emerging multidisciplinary area.

Sonification playback rates during matching tasks of visualised and sonified EEG data

In this paper, the authors discuss a user study examining the role of sonification in electroencephalography (EEG) data presentation. Conventionally, EEG data are presented using visualisation techniques and incorporate multivariate, time-critical information. As the number of EEG channels increase, or when screen real-estate is reduced, visually-presented data can become cluttered and occluded. Our user study examined how accurately users could match visualised EEG data to sonic equivalents, and at what playback rate this was most effective. Accuracy and timing data were recorded, as well as task load index (TLX) questionnaires. Results show that faster playback rates of sonified EEG data yield more accurate results. However, matching accuracy of sonified EEG data in the form presented in this study was not sufficient to replace visualized EEG. Although presently sonified electroencephalograms are not a complete replacement, sonification has the potential to effectively represent aspects of EEG data when visualisation alone becomes challenging for the user. The authors therefore propose a multimodal approach to EEG data presentation aimed at reducing visual clutter and reducing the cognitive load experienced by users when presented with too many dynamic variables on screen.

NEW SONIFICATION TOOLS FOR EEG DATA SCREENING AND MONITORING

This paper describes two software implementations for EEG data screening and realtime monitoring by means of sonification. Both have been designed in close collaboration with our partner institutions. Both tools were tested in depth with volunteers, and then tested with the expert users they are intended for, i.e. neurologists working with EEG data. In the course of these tests, a number of improvements to the designs were realised; tests and the final versions of the tools are described in detail. The scope of the paper is intended to provide an integrated description and analysis of all aspects of the design process from sonification design issues to interaction choices to user acceptance.

"Brain Listening" -A Sound Installation With EEG Sonification-

A sound installation using sonification of EEG (electroencephalogram, brain waves) is implemented based on a previous development in 2000. It consists of a wireless EEG headset, a PC with Max/MSP installed on it, and an audio interface with active speakers. Two Max/MSP patches for EEG sonification called "frequency modulation of sine wave" and "harmonic structure replication" are described.

Comparison of Visual and Auditory Modalities for Upper-Alpha EEG-Neurofeedback

2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019

Electroencephalography (EEG) neurofeedback (NF) training has been shown to produce long-lasting effects on the improvement of cognitive function as well as the normalization of aberrant brain activity in disease. However, the impact of the sensory modality used as the NF reinforcement signal on training effectiveness has not been systematically investigated. In this work, an EEG-based NFtraining system was developed targeting the individual upperalpha (UA) band and using either a visual or an auditory reinforcement signal, so as to compare the effects of the two sensory modalities. Sixteen healthy volunteers were randomly assigned to the Visual or Auditory group, where a radiusvarying sphere or a volume-varying sound, respectively, reflected the relative amplitude of UA measured at EEG electrode Cz. Each participant underwent a total of four NF sessions, of approximately 40 min each, on consecutive days. Both groups showed significant increases in UA at Cz within sessions, and also across sessions. Effects subsequent to NF training were also found beyond the target frequency UA and scalp location Cz, namely in the lower-alpha and theta bands and in posterior brain regions, respectively. Only small differences were found on the EEG between the Visual and Auditory groups, suggesting that auditory reinforcement signals may be as effective as the more commonly used visual signals. The use of auditory NF may potentiate training protocols conducted under mobile conditions, which are now possible due to the increasing availability of wireless EEG systems.

From auditory and visual to immersive neurofeedback: application to diagnosis of Alzheimer’s disease

In neurofeedback, brain waves are transformed into sounds or music, graphics, and other representations, to provide real-time information on ongoing waves and patterns in the brain. Here we present various forms of neurofeedback, including sonification, sonification in combination with visualization, and at last, immersive neurofeedback, where auditory and visual feedback is provided in a multi-sided immersive environment in which participants are completely 63 surrounded by virtual imagery and 3D sound. Neural feedback may potentially improve the user's (or patient's) ability to control brain activity, the diagnosis of medical conditions, and the rehabilitation of neurological or psychiatric disorders. Several psychological and medical studies have confirmed that virtual immersive activity is enjoyable, stimulating, and can have a healing effect. As an illustration, neurofeedback is generated from electroencephalograms (EEG) of Alzheimer's disease (AD) patients and healthy subjects. The auditory, visual, and immersive representations of Alzheimer's EEG differ substantially from healthy EEG, potentially yielding novel diagnostic tools. Moreover, such alternative representations of AD EEG are natural and intuitive, and hence easily accessible to laymen (AD patients and family members), and can provide insight into the abnormal brainwaves associated with AD.

Comparison of Visual and Auditory Modalities for Upper-Alpha EEG-Neurofeedback

2019

Electroencephalography (EEG) neurofeedback (NF) training has been shown to produce long-lasting effects on the improvement of cognitive function as well as the normalization of aberrant brain activity in disease. However, the impact of the sensory modality used as the NF reinforcement signal on training effectiveness has not been systematically investigated. In this work, an EEG-based NFtraining system was developed targeting the individual upperalpha (UA) band and using either a visual or an auditory reinforcement signal, so as to compare the effects of the two sensory modalities. Sixteen healthy volunteers were randomly assigned to the Visual or Auditory group, where a radiusvarying sphere or a volume-varying sound, respectively, reflected the relative amplitude of UA measured at EEG electrode Cz. Each participant underwent a total of four NF sessions, of approximately 40 min each, on consecutive days. Both groups showed significant increases in UA at Cz within sessions, and also ...

BSoniq: A 3-D EEG Sound Installation

2017

This paper presents “BSoniq”, a 3-D EEG sound installation, in which, the user can perceive spatial characteristics of EEG signals in a multi-channel environment. With this installation, the users (listeners) wear a wireless EEG headset and listen to sounds generated in real-time from their brain waves to perceive brain activities which they may not be aware of in their daily life. To accomplish a brain electrical activity sonification, brainwave source localization features of multichannel EEG are converted into sound images. These allow for simple interpretation, because of their spatial temporal differences. Signals recorded from the scalp are “decoded” from the multi-channel EEG, by applying filters and modulation to the EEG signal with an audio file. The main goal is to use sound to render the original data in a suitably transformed way so that we can invoke our natural pattern recognition capabilities to search for regularities and structures. Brainwave sonification is also ve...