Avinash Parnandi | NYU School of Medicine (original) (raw)

Papers by Avinash Parnandi

IEEE Journal of Biomedical and Health Informatics, 2015

We present an adaptive biofeedback game for teaching self-regulation of stress. Our approach cons... more We present an adaptive biofeedback game for teaching self-regulation of stress. Our approach consists of monitoring the user's physiology during gameplay and adapting the game using a positive feedback loop that rewards relaxing behaviors and penalizes states of high arousal. We evaluate the approach using a casual game under three biofeedback modalities: electrodermal activity, heart rate variability and breathing rate. The three biosignals can be measured non-invasively with wearable sensors, and represent different degrees of voluntary control and selectivity towards arousal. We conducted an experiment trial with 25 participants to compare the three modalities against a standard treatment (deep breathing) and a control condition (the game without biofeedback). Our results indicate that breathing-based game biofeedback is more effective in inducing relaxation during treatment than the other four groups. Participants in this group also showed greater retention of the relaxation skills (without biofeedback) during a subsequent stressor.

ABSTRACT We present a multi-tier system for the remote administration of speech therapy to childr... more ABSTRACT We present a multi-tier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical settings. Namely, the system allows a speech therapist to remotely assign speech production exercises to each child through a web interface, and the child to practice these exercises on a mobile device. The mobile app records the child's utterances and streams them to a back-end server for automated scoring by a speech-analysis engine. The therapist can then review the individual recordings and the automated scores through a web interface, provide feedback to the child, and adapt the training program as needed. We validated the system through a pilot study with children diagnosed with apraxia of speech, and their parents and speech therapists. Here we describe the overall client-server architecture, middleware tools used to build the system, the speech-analysis tools for automatic scoring of recorded utterances, and results from the pilot study. Our results support the feasibility of the system as a complement to traditional face-to-face therapy through the use of mobile tools and automated speech analysis algorithms.

ACM Transactions on Accessible Computing, 2015

Children with developmental disabilities such as childhood apraxia of speech (CAS) require repeat... more Children with developmental disabilities such as childhood apraxia of speech (CAS) require repeated intervention sessions with a speech therapist, sometimes extending over several years. Technology-based therapy tools offer the potential to reduce the demanding workload of speech therapists as well as time and cost for families. In response to this need, we have developed “Tabby Talks,” a multi-tier system for remote administration of speech therapy. This paper describes the speech processing pipeline to automatically detect common errors associated with CAS. The pipeline contains modules for voice activity detection, pronunciation verification, and lexical stress verification. The voice activity detector evaluates the intensity contour of an utterance and compares it against an adaptive threshold to detect silence segments and measure voicing delays and total production time. The pronunciation verification module uses a generic search lattice structure with multiple internal paths that covers all possible pronunciation errors (substitutions, insertions and deletions) in the child’s production. Finally, the lexical stress verification module classifies the lexical stress across consecutive syllables into strong–weak or weak-strong patterns using a combination of prosodic and spectral measures. These error measures can be provided to the therapist through a web interface, to enable them to adapt the child’s therapy program remotely. When evaluated on a dataset of typically developing and disordered speech from children ages 4–16 years, the system achieves a pronunciation verification accuracy of 88.2% at the phoneme level and 80.7% at the utterance level, and lexical stress classification rate of 83.3%.

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 2014

Proceedings of the Workshop on Performance Metrics for Intelligent Systems - PerMIS '12, 2012

Abstract This paper proposes a test method for measuring the ability of a USV or ROV to fixate on... more Abstract This paper proposes a test method for measuring the ability of a USV or ROV to fixate on an underwater object, ie, station-keeping. Station-keeping is needed to permit an operator or domain expert to stay focused on an area in an image long enough to identify objects, such as submerged cars and debris, or a condition, such as scour eroding the underwater footing of a bridge. This problem is different from traditional robot control, as the point is not to measure the positions of the robot and sensor payload but rather how well ...

Paladyn, 2011

Due to their quantitative nature, robotic systems are useful tools for systematically augmenting ... more Due to their quantitative nature, robotic systems are useful tools for systematically augmenting human behavior and performance in dynamic environments, such as therapeutic rehabilitation settings. The efficacy of human-robot interaction (HRI) in these settings will depend on the robot’s coaching style. Our goal was to investigate the influence of robot coaching styles designed to enhance motivation and encouragement on post-stroke individuals during motor task practice. We hypothesized that coaching styles incorporating user performance and preference would be preferred in a therapeutic HRI setting. We designed an evaluation study with seven individuals post stroke. A socially assistive robotics (SAR) system using three different coaching styles guided participants during performance of an upper extremity practice task. User preference was not significantly affected by the different robot coaching styles in our participant sample (H(2) = 2.638, p = 0.267). However, trends indicated...

Journal on Multimodal User Interfaces, 2014

ABSTRACT We present an adaptive biofeedback game that aims to maintain the player’s arousal by mo... more ABSTRACT We present an adaptive biofeedback game that aims to maintain the player’s arousal by modifying game difficulty in response to the player’s physiological state, as measured with wearable sensors. Our approach models the interaction between human physiology and game difficulty during gameplay as a control problem, where game difficulty is the system input and player arousal its output. We validate the approach on a car-racing game with real-time adaptive game mechanics. Specifically, we use (1) car speed, road visibility, and steering jitter as three mechanisms to manipulate game difficulty, (2) electrodermal activity as physiological correlate of arousal, and (3) two types of control law: proportional (P) control, and proportional-integral-derivative (PID) control. We also propose quantitative measures to characterize the effectiveness of these game adaptations and controllers in manipulating the player’s arousal. Experimental trials with 25 subjects in both open-loop (no feedback) and closed-loop (negative feedback) conditions show statistically significant differences in effectiveness among the three game mechanics and also between the two control laws. Specifically, manipulating car speed provides higher control of arousal levels than changing road visibility or vehicle steering. Our results also confirm that PID control leads to lower error and reduced oscillations in the closed-loop response compared to proportional-only control. Finally, we discuss the theoretical and practical implications of our approach.

Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS '13, 2013

ABSTRACT We present a multi-tier system for the remote administration of speech therapy to childr... more ABSTRACT We present a multi-tier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical settings. Namely, the system allows a speech therapist to remotely assign speech production exercises to each child through a web interface, and the child to practice these exercises on a mobile device. The mobile app records the child's utterances and streams them to a back-end server for automated scoring by a speech-analysis engine. The therapist can then review the individual recordings and the automated scores through a web interface, provide feedback to the child, and adapt the training program as needed. We validated the system through a pilot study with children diagnosed with apraxia of speech, and their parents and speech therapists. Here we describe the overall client-server architecture, middleware tools used to build the system, the speech-analysis tools for automatic scoring of recorded utterances, and results from the pilot study. Our results support the feasibility of the system as a complement to traditional face-to-face therapy through the use of mobile tools and automated speech analysis algorithms.

Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA '14, 2014

ABSTRACT We present an auditory biofeedback technique that may be used as a tool for stress manag... more ABSTRACT We present an auditory biofeedback technique that may be used as a tool for stress management. The technique encourages slow breathing by adjusting the quality of a music recording in proportion to the user's respiration rate. We propose two forms of acoustic degradation, one that adds white noise to the recording if the user's breathing deviates from the target rate, and another that reduces the number of channels in a multi-track recording. Validation on a small user study indicates that both techniques are equally effective at reducing respiration rates while performing a secondary task, though user feedback indicates that additive noise is a more intuitive form of sonification.

2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, 2013

ABSTRACT We present an adaptive biofeedback game that aims to maintain the player's arous... more ABSTRACT We present an adaptive biofeedback game that aims to maintain the player's arousal level by monitoring physiological signals. We use concepts from control theory to model the interaction between human physiology and game difficulty during game play. We validate the approach on a car-racing game with real-time adaptive game mechanics. Specifically, we use car speed, road visibility, and steering jitter as three mechanisms to manipulate game difficulty. We propose quantitative measures to characterize the effectiveness of these game adaptations in manipulating the player's arousal. For this purpose, we use electro dermal activity (EDA) as a physiological correlate of arousal. Experimental trials with 20 subjects in both open-loop (no feedback) and closed-loop (negative feedback) conditions show statistically significant differences among the three game mechanics in terms of their effectiveness. Specifically, manipulating car speed provides higher arousal levels than changing road visibility or vehicle steering. Finally, we discuss the theoretical and practical implications of our approach.

2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, 2013

ABSTRACT The ability to measure a person's physiological parameters in a contact less fas... more ABSTRACT The ability to measure a person's physiological parameters in a contact less fashion (i.e., without attaching electrodes to the skin) has tremendous potential in a number of applications, from affective interfaces to healthcare delivery. In this paper, we present a proof-of-concept method for measuring one such vital parameter, heart rate variability (HRV), in a contact less fashion from spontaneous fluctuations in pupillary diameter. Our approach uses a remote eye tracker for imaging and an integro-differential algorithm for segmenting the pupil-iris boundary. We then estimate HRV from the relative distribution of energy in the low frequency (0.04 to 0.15 Hz) and high frequency (0.15 to 0.4 Hz) bands of the power spectrum of the time series of pupillary fluctuations. We validated the method under a range of breathing conditions and under different illumination levels. Our results show a high degree of agreement between our pupillary estimate of HRV and ground truth measurements from an ECG-grade heart rate monitor. These results support the feasibility of estimating HRV in a non-contact, non-invasive fashion.

Abstract—We present an approach to wearable sensor-based assessment of functional ability (FA) of... more Abstract—We present an approach to wearable sensor-based assessment of functional ability (FA) of individuals post stroke. We make use of one on-body inertial measurement unit (IMU) to automate the FA scoring of the Wolf Motor Function Test (WMFT). WMFT is an assessment instrument used to determine the functional capabilities of individuals post stroke. It comprises of 17 tasks-15 of which are rated according to performance time and quality of motion.

Abstract The advent of new health sensing technologies has presented us with the opportunity to g... more Abstract The advent of new health sensing technologies has presented us with the opportunity to gain richer data from patients undergoing clinical interventions. Such technologies are particularly suited for applications requiring temporal accuracy. The Wolf Motor Function Test (WMFT) is one such application. This assessment is an instrument used to determine functional ability of the paretic and non-paretic limbs in individuals post-stroke.

IEEE Journal of Biomedical and Health Informatics, 2015

We present an adaptive biofeedback game for teaching self-regulation of stress. Our approach cons... more We present an adaptive biofeedback game for teaching self-regulation of stress. Our approach consists of monitoring the user's physiology during gameplay and adapting the game using a positive feedback loop that rewards relaxing behaviors and penalizes states of high arousal. We evaluate the approach using a casual game under three biofeedback modalities: electrodermal activity, heart rate variability and breathing rate. The three biosignals can be measured non-invasively with wearable sensors, and represent different degrees of voluntary control and selectivity towards arousal. We conducted an experiment trial with 25 participants to compare the three modalities against a standard treatment (deep breathing) and a control condition (the game without biofeedback). Our results indicate that breathing-based game biofeedback is more effective in inducing relaxation during treatment than the other four groups. Participants in this group also showed greater retention of the relaxation skills (without biofeedback) during a subsequent stressor.

ABSTRACT We present a multi-tier system for the remote administration of speech therapy to childr... more ABSTRACT We present a multi-tier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical settings. Namely, the system allows a speech therapist to remotely assign speech production exercises to each child through a web interface, and the child to practice these exercises on a mobile device. The mobile app records the child's utterances and streams them to a back-end server for automated scoring by a speech-analysis engine. The therapist can then review the individual recordings and the automated scores through a web interface, provide feedback to the child, and adapt the training program as needed. We validated the system through a pilot study with children diagnosed with apraxia of speech, and their parents and speech therapists. Here we describe the overall client-server architecture, middleware tools used to build the system, the speech-analysis tools for automatic scoring of recorded utterances, and results from the pilot study. Our results support the feasibility of the system as a complement to traditional face-to-face therapy through the use of mobile tools and automated speech analysis algorithms.

ACM Transactions on Accessible Computing, 2015

Children with developmental disabilities such as childhood apraxia of speech (CAS) require repeat... more Children with developmental disabilities such as childhood apraxia of speech (CAS) require repeated intervention sessions with a speech therapist, sometimes extending over several years. Technology-based therapy tools offer the potential to reduce the demanding workload of speech therapists as well as time and cost for families. In response to this need, we have developed “Tabby Talks,” a multi-tier system for remote administration of speech therapy. This paper describes the speech processing pipeline to automatically detect common errors associated with CAS. The pipeline contains modules for voice activity detection, pronunciation verification, and lexical stress verification. The voice activity detector evaluates the intensity contour of an utterance and compares it against an adaptive threshold to detect silence segments and measure voicing delays and total production time. The pronunciation verification module uses a generic search lattice structure with multiple internal paths that covers all possible pronunciation errors (substitutions, insertions and deletions) in the child’s production. Finally, the lexical stress verification module classifies the lexical stress across consecutive syllables into strong–weak or weak-strong patterns using a combination of prosodic and spectral measures. These error measures can be provided to the therapist through a web interface, to enable them to adapt the child’s therapy program remotely. When evaluated on a dataset of typically developing and disordered speech from children ages 4–16 years, the system achieves a pronunciation verification accuracy of 88.2% at the phoneme level and 80.7% at the utterance level, and lexical stress classification rate of 83.3%.

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 2014

Proceedings of the Workshop on Performance Metrics for Intelligent Systems - PerMIS '12, 2012

Abstract This paper proposes a test method for measuring the ability of a USV or ROV to fixate on... more Abstract This paper proposes a test method for measuring the ability of a USV or ROV to fixate on an underwater object, ie, station-keeping. Station-keeping is needed to permit an operator or domain expert to stay focused on an area in an image long enough to identify objects, such as submerged cars and debris, or a condition, such as scour eroding the underwater footing of a bridge. This problem is different from traditional robot control, as the point is not to measure the positions of the robot and sensor payload but rather how well ...

Paladyn, 2011

Due to their quantitative nature, robotic systems are useful tools for systematically augmenting ... more Due to their quantitative nature, robotic systems are useful tools for systematically augmenting human behavior and performance in dynamic environments, such as therapeutic rehabilitation settings. The efficacy of human-robot interaction (HRI) in these settings will depend on the robot’s coaching style. Our goal was to investigate the influence of robot coaching styles designed to enhance motivation and encouragement on post-stroke individuals during motor task practice. We hypothesized that coaching styles incorporating user performance and preference would be preferred in a therapeutic HRI setting. We designed an evaluation study with seven individuals post stroke. A socially assistive robotics (SAR) system using three different coaching styles guided participants during performance of an upper extremity practice task. User preference was not significantly affected by the different robot coaching styles in our participant sample (H(2) = 2.638, p = 0.267). However, trends indicated...

Journal on Multimodal User Interfaces, 2014

ABSTRACT We present an adaptive biofeedback game that aims to maintain the player’s arousal by mo... more ABSTRACT We present an adaptive biofeedback game that aims to maintain the player’s arousal by modifying game difficulty in response to the player’s physiological state, as measured with wearable sensors. Our approach models the interaction between human physiology and game difficulty during gameplay as a control problem, where game difficulty is the system input and player arousal its output. We validate the approach on a car-racing game with real-time adaptive game mechanics. Specifically, we use (1) car speed, road visibility, and steering jitter as three mechanisms to manipulate game difficulty, (2) electrodermal activity as physiological correlate of arousal, and (3) two types of control law: proportional (P) control, and proportional-integral-derivative (PID) control. We also propose quantitative measures to characterize the effectiveness of these game adaptations and controllers in manipulating the player’s arousal. Experimental trials with 25 subjects in both open-loop (no feedback) and closed-loop (negative feedback) conditions show statistically significant differences in effectiveness among the three game mechanics and also between the two control laws. Specifically, manipulating car speed provides higher control of arousal levels than changing road visibility or vehicle steering. Our results also confirm that PID control leads to lower error and reduced oscillations in the closed-loop response compared to proportional-only control. Finally, we discuss the theoretical and practical implications of our approach.

Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS '13, 2013

ABSTRACT We present a multi-tier system for the remote administration of speech therapy to childr... more ABSTRACT We present a multi-tier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical settings. Namely, the system allows a speech therapist to remotely assign speech production exercises to each child through a web interface, and the child to practice these exercises on a mobile device. The mobile app records the child's utterances and streams them to a back-end server for automated scoring by a speech-analysis engine. The therapist can then review the individual recordings and the automated scores through a web interface, provide feedback to the child, and adapt the training program as needed. We validated the system through a pilot study with children diagnosed with apraxia of speech, and their parents and speech therapists. Here we describe the overall client-server architecture, middleware tools used to build the system, the speech-analysis tools for automatic scoring of recorded utterances, and results from the pilot study. Our results support the feasibility of the system as a complement to traditional face-to-face therapy through the use of mobile tools and automated speech analysis algorithms.

Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA '14, 2014

ABSTRACT We present an auditory biofeedback technique that may be used as a tool for stress manag... more ABSTRACT We present an auditory biofeedback technique that may be used as a tool for stress management. The technique encourages slow breathing by adjusting the quality of a music recording in proportion to the user's respiration rate. We propose two forms of acoustic degradation, one that adds white noise to the recording if the user's breathing deviates from the target rate, and another that reduces the number of channels in a multi-track recording. Validation on a small user study indicates that both techniques are equally effective at reducing respiration rates while performing a secondary task, though user feedback indicates that additive noise is a more intuitive form of sonification.

2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, 2013

ABSTRACT We present an adaptive biofeedback game that aims to maintain the player's arous... more ABSTRACT We present an adaptive biofeedback game that aims to maintain the player's arousal level by monitoring physiological signals. We use concepts from control theory to model the interaction between human physiology and game difficulty during game play. We validate the approach on a car-racing game with real-time adaptive game mechanics. Specifically, we use car speed, road visibility, and steering jitter as three mechanisms to manipulate game difficulty. We propose quantitative measures to characterize the effectiveness of these game adaptations in manipulating the player's arousal. For this purpose, we use electro dermal activity (EDA) as a physiological correlate of arousal. Experimental trials with 20 subjects in both open-loop (no feedback) and closed-loop (negative feedback) conditions show statistically significant differences among the three game mechanics in terms of their effectiveness. Specifically, manipulating car speed provides higher arousal levels than changing road visibility or vehicle steering. Finally, we discuss the theoretical and practical implications of our approach.

2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, 2013

ABSTRACT The ability to measure a person's physiological parameters in a contact less fas... more ABSTRACT The ability to measure a person's physiological parameters in a contact less fashion (i.e., without attaching electrodes to the skin) has tremendous potential in a number of applications, from affective interfaces to healthcare delivery. In this paper, we present a proof-of-concept method for measuring one such vital parameter, heart rate variability (HRV), in a contact less fashion from spontaneous fluctuations in pupillary diameter. Our approach uses a remote eye tracker for imaging and an integro-differential algorithm for segmenting the pupil-iris boundary. We then estimate HRV from the relative distribution of energy in the low frequency (0.04 to 0.15 Hz) and high frequency (0.15 to 0.4 Hz) bands of the power spectrum of the time series of pupillary fluctuations. We validated the method under a range of breathing conditions and under different illumination levels. Our results show a high degree of agreement between our pupillary estimate of HRV and ground truth measurements from an ECG-grade heart rate monitor. These results support the feasibility of estimating HRV in a non-contact, non-invasive fashion.

Abstract—We present an approach to wearable sensor-based assessment of functional ability (FA) of... more Abstract—We present an approach to wearable sensor-based assessment of functional ability (FA) of individuals post stroke. We make use of one on-body inertial measurement unit (IMU) to automate the FA scoring of the Wolf Motor Function Test (WMFT). WMFT is an assessment instrument used to determine the functional capabilities of individuals post stroke. It comprises of 17 tasks-15 of which are rated according to performance time and quality of motion.

Abstract The advent of new health sensing technologies has presented us with the opportunity to g... more Abstract The advent of new health sensing technologies has presented us with the opportunity to gain richer data from patients undergoing clinical interventions. Such technologies are particularly suited for applications requiring temporal accuracy. The Wolf Motor Function Test (WMFT) is one such application. This assessment is an instrument used to determine functional ability of the paretic and non-paretic limbs in individuals post-stroke.