Vesna Novak | University of Cincinnati (original) (raw)

Papers by Vesna Novak

Proceedings of the 2023 IEEE International Conference on Rehabilitation Robotics, 2023

Rehabilitation after neurological injury can be provided by robots that help patients perform dif... more Rehabilitation after neurological injury can be provided by robots that help patients perform different exercises. Multiple such robots can be combined in a rehabilitation robot gym to allow multiple patients to perform a diverse range of exercises simultaneously. In pursuit of better multipatient supervision, we aim to develop an automated assignment system that assigns patients to different robots during a training session to maximize their skill development. Our previous work was designed for simplified simulated environments where each patient's skill development is known beforehand. The current work improves upon that work by changing the deterministic environment into a stochastic environment where part of the skill development is random and the assignment system must estimate each patient's predicted skill development using a neural network based on the patient's previous training success rate with that robot. These skill development estimates are used to create patient-robot assignments on a timestep-by-timestep basis to maximize the skill development of the patient group. Results from simplified simulation trials show that the schedules produced by our assignment system outperform multiple baseline schedules (e.g., schedules where patients never switch robots and schedules where patients only switch robots once halfway through the session). Additionally, we discuss how some of our simplifications could be addressed in the future.

Proceedings of the 45th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2023

Passive back support exoskeletons, which support the human trunk using elements like springs and ... more Passive back support exoskeletons, which support the human trunk using elements like springs and elastic bands, have demonstrated positive results in laboratory-based studies, but have seen significantly less field testing. As an intermediate step between generic lab evaluations and field tests, we conducted a single-session lab evaluation of the HeroWear Apex exoskeleton with mockup construction tasks: 20 adult men (without extensive construction experience) lifted, carried and raised lumber boards (265 cm length, up to 18 kg total load). The exoskeleton significantly reduced mean erector spinae electromyograms, with effect sizes (Cohen’s d) ranging from -0.2 to -0.55 – corresponding to reductions of 5-25% relative to no-exoskeleton electromyogram values. In asymmetric carrying tasks, the exoskeleton provided more assistance to the more heavily loaded erector spinae muscle. Additionally, in lifting tasks, the exoskeleton decreased trunk/hip flexion/extension range of motion and increased knee range of motion, indicating changes in lifting strategy. These results indicate potential exoskeleton benefits for lumber board carrying and will serve as the basis for further evaluations with workers in the field.

Journal of Biomechanics, 2023

Low-cost exoskeletons can effectively support workers in physically demanding jobs, but most such... more Low-cost exoskeletons can effectively support workers in physically demanding jobs, but most such exoskeletons have been developed to support repetitive lifting or uncomfortable static postures. Very few low-cost exoskeletons have been designed to support walking while carrying heavy objects, which would be beneficial for jobs such as moving furniture and warehouse work. This paper thus presents a single-session lab evaluation of the Auxivo CarrySuit, a low-cost upperbody exoskeleton designed for carrying objects that would normally be held with the arms. Twenty participants carried four loads (box or two bags, 20 or 40 lb total weight) for 2 minutes each on a treadmill with and without the CarrySuit. Across all loads, the CarrySuit significantly reduced the mean electromyogram of the middle trapezius (partial eta-squared = .74from 16.1% to 8.8% of maximum voluntary contraction value) and anterior deltoid (partial eta-squared = .26from 3.0% to 1.1% of maximum voluntary contraction value) with no corresponding increase in lower back muscle activation. Furthermore, maximum heart rate and Ratings of Perceived Exertion were also reduced by the CarrySuit, and discomfort was shifted from the upper body to the legs. While arm EMG was not measured, it is likely that it was also reduced due to the unloading of the arms. The CarrySuit can thus be considered beneficial in the short term, though longer-term evaluations with actual workers are needed to determine practical benefits.

IEEE Transactions on Affective Computing, 2023

Two people's physiological responses become more similar as those people talk or cooperate, a phe... more Two people's physiological responses become more similar as those people talk or cooperate, a phenomenon called physiological synchrony. The degree of synchrony correlates with conversation engagement and cooperation quality, and could thus be used to characterize interpersonal interaction. In this study, we used a combination of physiological synchrony metrics and pattern recognition algorithms to automatically classify four different dyadic conversation scenarios: two-sided positive conversation, two-sided negative conversation, and two one-sided scenarios. Heart rate, skin conductance, respiration and peripheral skin temperature were measured from 16 dyads in all four scenarios, and individual as well as synchrony features were extracted from them. A two-stage classifier based on stepwise feature selection and linear discriminant analysis achieved a four-class classification accuracy of 75.0% in leave-dyad-out crossvalidation. Removing synchrony features reduced accuracy to 65.6%, indicating that synchrony is informative. In the future, such classification algorithms may be used to, e.g., provide realtime feedback about conversation mood to participants, with applications in areas such as mental health counseling and education. The approach may also generalize to group scenarios and adjacent areas such as cooperation and competition.

Journal of NeuroEngineering and Rehabilitation, 2022

Background: A robotic rehabilitation gym can be defined as multiple patients training with multip... more Background: A robotic rehabilitation gym can be defined as multiple patients training with multiple robots or passive sensorized devices in a group setting. Recent work with such gyms has shown positive rehabilitation outcomes; furthermore, such gyms allow a single therapist to supervise more than one patient, increasing cost-effectiveness. To allow more effective multipatient supervision in future robotic rehabilitation gyms, we propose an automated system that could dynamically assign patients to different robots within a session in order to optimize rehabilitation outcome. Methods: As a first step toward implementing a practical patient-robot assignment system, we present a simplified mathematical model of a robotic rehabilitation gym. Mixed-integer nonlinear programming algorithms are used to find effective assignment and training solutions for multiple evaluation scenarios involving different numbers of patients and robots (5 patients and 5 robots, 6 patients and 5 robots, 5 patients and 7 robots), different training durations (7 or 12 time steps) and different complexity levels (whether different patients have different skill acquisition curves, whether robots have exit times associated with them). In all cases, the goal is to maximize total skill gain across all patients and skills within a session. Results: Analyses of variance across different scenarios show that disjunctive and time-indexed optimization models significantly outperform two baseline schedules: staying on one robot throughout a session and switching robots halfway through a session. The disjunctive model results in higher skill gain than the time-indexed model in the given scenarios, and the optimization duration increases as the number of patients, robots and time steps increases. Additionally, we discuss how different model simplifications (e.g., perfectly known and predictable patient skill level) could be addressed in the future and how such software may eventually be used in practice. Conclusions: Though it involves unrealistically simple scenarios, our study shows that intelligently moving patients between different rehabilitation robots can improve overall skill acquisition in a multi-patient multi-robot environment. While robotic rehabilitation gyms are not yet commonplace in clinical practice, prototypes of them already exist, and our study presents a way to use intelligent decision support to potentially enable more efficient delivery of technologically aided rehabilitation.

2022 International Conference on Rehabilitation Robotics (ICORR), 2022

A robotic rehabilitation gym is a setup that allows multiple patients to exercise together using ... more A robotic rehabilitation gym is a setup that allows multiple patients to exercise together using multiple robots. The effectiveness of training in such a group setting could be increased by dynamically assigning patients to specific robots. In this simulation study, we develop an automated system that dynamically makes patient-robot assignments based on measured patient performance to achieve optimal group rehabilitation outcome. To solve the dynamic assignment problem, we propose an approach that uses a neural network classifier to predict the assignment priority between two patients for a specific robot given their task success rate on that robot. The priority classifier is trained using assignment demonstrations provided by a domain expert. In the absence of real human data from a robotic gym, we develop a robotic gym simulator and create a synthetic dataset for training the classifier. The simulation results show that our approach makes effective assignments that yield comparable patient training outcomes to those obtained by the domain expert.

Journal of Voice

Transgender and gender diverse people often experience voice-gender incongruence, which is invers... more Transgender and gender diverse people often experience voice-gender incongruence, which is inversely correlated with health and quality of life. Such incongruence could be reduced with voice and communication training, but expert-administered training is often inaccessible while self-guided training is difficult and potentially risky. Training could alternatively be provided through software (eg, smartphone apps), but such software is at an early stage. This qualitatively driven mixed-methods study thus includes surveys and interviews with 21 transfeminine, transmasculine and nonbinary people to identify general views of voice and communication training software as well as most desirable features of such software. Participants were positive about the general idea and described ways to effectively implement four critical features: feedback, accountability, automated goal setting, and training characteristics other than pitch. They also discussed optional or undesirable features. These findings may inform development of voice and communication training software, thus improving health and quality of life for gender minorities.

Applied Ergonomics, 2022

Back support exosuits can support workers in physically demanding jobs by reducing muscle load, w... more Back support exosuits can support workers in physically demanding jobs by reducing muscle load, which could reduce risk of work-related musculoskeletal disorders. This paper presents a two-session evaluation of a commercial exosuit, the Auxivo LiftSuit 1.1. In session 1, 17 participants performed single repetitions of lifting and static leaning tasks with and without the LiftSuit. In session 2, 10 participants performed 50 box lifting repetitions with and without the LiftSuit. In session 1, the exosuit was considered mildly to moderately helpful, and reduced erector spinae and middle trapezius electromyograms. In session 2, the exosuit was not considered helpful, but reduced the middle trapezius electromyogram and trunk and thigh ranges of motion. These effects are likely due to placement of elastic elements and excessive stiffness at the hips. Overall, the LiftSuit appears suboptimal for long-term use, though elastic elements on the upper back may reduce muscle activation in future exosuit designs.

Frontiers in Neuroscience, 2021

Physiological responses of two interacting individuals contain a wealth of information about the ... more Physiological responses of two interacting individuals contain a wealth of information about the dyad: for example, the degree of engagement or trust. However, nearly all studies on dyadic physiological responses have targeted group-level analysis: e.g., correlating physiology and engagement in a large sample. Conversely, this paper presents a study where physiological measurements are combined with machine learning algorithms to dynamically estimate the engagement of individual dyads. Sixteen dyads completed 15-min naturalistic conversations and self-reported their engagement on a visual analog scale every 60 s. Four physiological signals (electrocardiography, skin conductance, respiration, skin temperature) were recorded, and both individual physiological features (e.g., each participant’s heart rate) and synchrony features (indicating degree of physiological similarity between two participants) were extracted. Multiple regression algorithms were used to estimate self-reported engagement based on physiological features using either leave-interval-out crossvalidation (training on 14 60-s intervals from a dyad and testing on the 15th interval from the same dyad) or leave-dyad-out crossvalidation (training on 15 dyads and testing on the 16th). In leave-interval-out crossvalidation, the regression algorithms achieved accuracy similar to a ‘baseline’ estimator that simply took the median engagement of the other 14 intervals. In leave-dyad-out crossvalidation, machine learning achieved a slightly higher accuracy than the baseline estimator and higher accuracy than an independent human observer. Secondary analyses showed that removing synchrony features and personality characteristics from the input dataset negatively impacted estimation accuracy and that engagement estimation error was correlated with personality traits. Results demonstrate the feasibility of dynamically estimating interpersonal engagement during naturalistic conversation using physiological measurements, which has potential applications in both conversation monitoring and conversation enhancement. However, as many of our estimation errors are difficult to contextualize, further work is needed to determine acceptable estimation accuracies.

Journal of Biomechanics, 2021

Trunk exoskeletons are wearable devices that support humans during physically demanding tasks by ... more Trunk exoskeletons are wearable devices that support humans during physically demanding tasks by reducing biomechanical loads on the back. While most trunk exoskeletons are rigid devices, more lightweight soft exoskeletons (exosuits) have recently been developed. One such exosuit is the HeroWear Apex, which achieved promising results in the developers’ own work but has not been independently evaluated. This paper thus presents an evaluation of the Apex with 20 adult participants during multiple brief tasks: standing up from a stool with a symmetric or asymmetric load, lifting a unilateral or bilateral load from the floor to waist level, lifting the same bilateral load with a 90-degree turn to the right, lowering a bilateral load from waist level to floor, and walking while carrying a bilateral load. The tasks were performed in an ABA-style protocol: first with exosuit assistance disengaged, then with it engaged, then disengaged again. Four measurement types were taken: electromyography (of the erector spinae, rectus abdominis, and middle trapezius), trunk kinematics, self-report ratings, and heart rate. The exosuit decreased the erector spinae electromyogram by about 15% during object lifting and lowering tasks; furthermore, participants found the exosuit mildly to moderately helpful. No adverse effects on other muscles or during non-lifting tasks were noted, and a decrease in middle trapezius electromyogram was observed for one task. This confirms that the HeroWear Apex could reduce muscle demand and fatigue. The results may transfer to other exoskeletons with similar design principles, and may inform researchers working with other wearable devices.

Sensors

Lifting and carrying heavy objects is a major aspect of physically intensive jobs. Wearable senso... more Lifting and carrying heavy objects is a major aspect of physically intensive jobs. Wearable sensors have previously been used to classify different ways of picking up an object, but have seen only limited use for automatic classification of load position and weight while a person is walking and carrying an object. In this proof-of-concept study, we thus used wearable inertial and electromyographic sensors for offline classification of different load positions (frontal vs. unilateral vs. bilateral side loads) and weights during gait. Ten participants performed 19 different carrying trials each while wearing the sensors, and data from these trials were used to train and evaluate classification algorithms based on supervised machine learning. The algorithms differentiated between frontal and other loads (side/none) with an accuracy of 100%, between frontal vs. unilateral side load vs. bilateral side load with an accuracy of 96.1%, and between different load asymmetry levels with accura...

International Journal of Human-Computer Studies, 2021

In competitive and cooperative scenarios, task difficulty should be dynamically adapted to suit p... more In competitive and cooperative scenarios, task difficulty should be dynamically adapted to suit people with different abilities. State-of-the-art difficulty adaptation methods for such scenarios are based on task performance, which conveys little information about user-specific factors such as workload. Thus, we present an exploratory study of automated affect recognition and task difficulty adaptation in a competitive scenario based on physiological linkage (covariation of participants’ physiological responses). Classification algorithms were developed in an open-loop study where 16 pairs played a competitive game while 5 physiological responses were measured: respiration, skin conductance, electrocardiogram, and 2 facial electromyograms. Physiological and performance data were used to classify four self-reported variables (enjoyment, valence, arousal, perceived difficulty) into two or three classes. The highest classification accuracies were obtained for perceived difficulty: 84.3% for two-class and 60.5% for three-class classification. As a proof of concept, the developed classifiers were used in a small closed-loop study to dynamically adapt game difficulty. While this closed-loop study found no clear advantages of physiology-based adaptation, it demonstrated the technical feasibility of such real-time adaptation. In the long term, physiology-based task adaptation could enhance competition and cooperation in many multi-user settings (e.g., education, manufacturing, exercise).

Presence: Virtual and Augmented Reality, 2020

In recent years, several multi-user virtual environments (VEs) have been developed to promote mot... more In recent years, several multi-user virtual environments (VEs) have been developed to promote motivation and exercise intensity in motor rehabilitation. While competitive VEs have been extensively evaluated, collaborative and competitive rehabilitation VEs have seen relatively little study. Therefore, this paper presents an evaluation of a VE for post-stroke arm rehabilitation that mimics everyday kitchen tasks and can be used either solo or collaboratively. Twenty subacute stroke survivors exercised with the VE for four sessions, with the first and third sessions involving solo exercise and the other two involving collaborative exercise. Exercise intensity was measured using inertial sensors while motivation was measured with questionnaires. Results showed high motivation and exercise intensity over all four sessions, and 11 of 20 participants preferred collaborative over solo exercise while only 4 preferred solo exercise.However, there were no differences in motivation, exercise duration, or exercise intensity between solo and collaborative sessions. Thus,we cannot currently claim that collaborative exercises are beneficial for upper limb rehabilitation. Future studies should evaluate other collaborative VE designs in different settings (e.g., at home) and with different participant pairs (e.g., patient-unimpaired) to find effective ways to utilize collaborative exercises in motor rehabilitation.

IEEE Transactions on Biomedical Engineering, 2020

Objective: Trunk exoskeletons are a new technology with great promise for human rehabilitation, a... more Objective: Trunk exoskeletons are a new technology with great promise for human rehabilitation, assistance and augmentation. However, it is unclear how different exoskeleton features affect the wearer's body during different activities. This study thus examined how varying a trunk exoskeleton's thoracic and abdominal compression affects trunk kinematics and muscle demand during several activities. Methods: We developed a trunk exoskeleton that allows thoracic and abdominal compression to be changed quickly and independently. To evaluate the effect of varying compression, 12 participants took part in a two-session study. In the first session, they performed three activities (walking, sit-to-stand, lifting a box). In the second session, they experienced unexpected perturbations while sitting. This was done both without the exoskeleton and in four exoskeleton configurations with different thoracic and abdominal compression levels. Trunk flexion angle, low back extension moment and the electromyogram of the erector spinae and rectus abdominis were measured in both sessions. Results: Different exoskeleton compression levels resulted in significantly different peak trunk flexion angles and peak electromyograms of the erector spinae. However, the effects of compression differed significantly between activities. Conclusion: Our results indicate that a trunk exoskeleton's thoracic and abdominal compression affect the wearer's kinematics and muscle demand; furthermore, a single compression configuration is not appropriate for all activities. Significance: The study suggests that future trunk exoskeletons should either be able to vary their compression levels to suit different activities or should have the compression designed for a specific activity in order to be beneficial to the wearer.

Sensors, 2020

Although several studies have used wearable sensors to analyze human lifting, this has generally ... more Although several studies have used wearable sensors to analyze human lifting, this has generally only been done in a limited manner. In this proof-of-concept study, we investigate multiple aspects of offline lift characterization using wearable inertial measurement sensors: detecting the start and end of the lift and classifying the vertical movement of the object, the posture used, the weight of the object, and the asymmetry involved. In addition, the lift duration, horizontal distance from the lifter to the object, the vertical displacement of the object, and the asymmetric angle are computed as lift parameters. Twenty-four healthy participants performed two repetitions of 30 different main lifts each while wearing a commercial inertial measurement system. The data from these trials were used to develop, train, and evaluate the lift characterization algorithms presented. The lift detection algorithm had a start time error of 0.10 s ± 0.21 s and an end time error of 0.36 s ± 0.27 s across all 1489 lift trials with no missed lifts. For posture, asymmetry, vertical movement, and weight, our classifiers achieved accuracies of 96.8%, 98.3%, 97.3%, and 64.2%, respectively, for automatically detected lifts. The vertical height and displacement estimates were, on average, within 25 cm of the reference values. The horizontal distances measured for some lifts were quite different than expected (up to 14.5 cm), but were very consistent. Estimated asymmetry angles were similarly precise. In the future, these proof-of-concept offline algorithms can be expanded and improved to work in real-time. This would enable their use in applications such as real-time health monitoring and feedback for assistive devices.

42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2020

In dyadic motor learning, pairs of people learn the same motion while their limbs are loosely cou... more In dyadic motor learning, pairs of people learn the same motion while their limbs are loosely coupled together using haptic devices. Such coupled learning has been shown to outperform solo learning (including robot-guided learning) for simple one-degree-of-freedom tasks. However, results from more complex tasks are limited and sometimes conflicting. We thus evaluated coupled learning in a two-degree-of-freedom tracking task where participants also had to compensate for a simple force field. Participant pairs were split into two groups: an experiment group that experienced a compliant haptic coupling between participants' hands and a control group that did not. The study protocol consisted of 70 repetitions of 18.9-second tracking trials: 10 initial solo trials with no coupling, 50 "learning" trials (where participants in the experiment group were coupled), and 10 final solo trials with no coupling. The experiment group (coupled) improved their solo tracking performance both in the presence (p = 0.008) and absence (p < 0.001) of the force field; however, the control group (no coupling) only improved their solo performance in the absence of the force field (p < 0.001) but not in the presence of the field (p = 0.81). This suggests that dyadic motor learning can outperform solo learning for two-dimensional tracking motions in the presence of a simple force field, though the mechanism by which learning is improved is not yet clear. Clinical Relevance-As motor learning is critical for applications such as motor rehabilitation, dyadic training could be used to achieve a better overall outcome and a faster learning speed in these applications compared to solo training.

Games for Health Journal, 2020

Objective: Competitive exercise games are popular in areas like rehabilitation and weight loss du... more Objective: Competitive exercise games are popular in areas like rehabilitation and weight loss due to their positive effects on motivation. However, it is unclear whether a human opponent is necessary, as the same benefits may be achievable with a ''human-like'' computer-controlled opponent or a human who talks to the player without playing the game. Our objective was to compare four opponent types in a competitive exercise game: a simple computer opponent, ''human-like'' computer opponent, human opponent, and a simple computer opponent accompanied by a player-selected human who chats with the player. Materials and Methods: Sixteen participants (3 women, 24.4-7.7 years old) played a competitive arm exercise game in the above four conditions. Exercise intensity was measured with inertial sensors, and four motivation scales were measured with the Intrinsic Motivation Inventory. After playing, participants answered several questions regarding their preferences. Results: The human opponent was the favorite for 14 of 16 participants and resulted in the highest interest/ enjoyment and exercise intensity. All participants preferred the human opponent over the computer opponent accompanied by a human companion. Finally, 12 of 16 participants preferred the ''human-like'' computer opponent over the simple one. Conclusion: Our results have two implications for competitive exercise games. First, they indicate that developing computer-controlled opponents with more human-like behavior is worthwhile, but that the best results are achieved with human opponents. Second, social interaction without in-game interaction does not provide an enjoyable, intense experience. However, our results should be verified with different target populations for exercise games.

JMIR Serious Games, 2019

Background: Competitive and cooperative serious games have become increasingly popular in areas s... more Background: Competitive and cooperative serious games have become increasingly popular in areas such as rehabilitation and education and have several potential advantages over single-player games. However, they are not suitable for everyone, and the user experience in competitive and cooperative serious games depends on many factors. One important factor is the verbal interaction between players, but the effect of this factor has not been extensively studied because of the lack of a validated measurement tool.

Frontiers in Neuroscience, 2019

Human psychological (cognitive and affective) dimensions can be assessed using several methods, s... more Human psychological (cognitive and affective) dimensions can be assessed using several methods, such as physiological or performance measurements. To date, however, few studies have compared different data modalities with regard to their ability to enable accurate classification of different psychological dimensions. This study thus compares classification accuracies for four psychological dimensions and two subjective preferences about computer game difficulty using three data modalities: physiology, performance, and personality characteristics. Thirty participants played a computer game at nine difficulty configurations that were implemented via two difficulty parameters. In each configuration, seven physiological measurements and two performance variables were recorded. A short questionnaire was filled out to assess the perceived difficulty, enjoyment, valence, arousal, and the way the participant would like to modify the two difficulty parameters. Furthermore, participants' personality characteristics were assessed using four questionnaires. All combinations of the three data modalities (physiology, performance, and personality) were used to classify six dimensions of the short questionnaire into either two, three or many classes using four classifier types: linear discriminant analysis, support vector machine (SVM), ensemble decision tree, and multiple linear regression. The classification accuracy varied widely between the different psychological dimensions; the highest accuracies for two-class and three-class classification were 97.6 and 84.1%, respectively. Normalized physiological measurements were the most informative data modality, though current game difficulty, personality and performance also contributed to classification accuracy; the best selected features are presented and discussed in the text. The SVM and multiple linear regression were the most accurate classifiers, with regression being more effective for normalized physiological data. In the future, we will further examine the effect of different classification approaches on user experience by detecting the user's psychological state and adapting game difficulty in real-time. This will allow us to obtain a complete picture of the performance of affect-aware systems in both an offline (classification accuracy) and real-time (effect on user experience) fashion.

2019 IEEE International Conference on Rehabilitation Robotics, 2019

Interpersonal rehabilitation games, which allow patients to compete or cooperate with other patie... more Interpersonal rehabilitation games, which allow patients to compete or cooperate with other patients or unimpaired loved ones, have demonstrated promising short-term results, but have not yet been tested in longer-term studies. This paper thus presents a preliminary 9-session evaluation of interpersonal rehabilitation games for post-stroke arm exercise. Two pairs of stroke survivors were provided with a system that included one competitive and one cooperative rehabilitation game, and exercised with it for 9 sessions in addition to their conventional therapy. They were able to choose the game they wanted to play in each session, and had to exercise for at least 10 minutes per session. Both pairs completed the protocol without any issues, reporting high levels of motivation and consistent levels of exercise intensity (measured using inertial sensors) across the sessions. Furthermore, the maximum difficulty levels reached in the cooperative game increased over time, and improvements of 1-8 points were observed on the Box and Block test. These results indicate that 2 different interpersonal games are sufficient to promote high levels of motivation and exercise intensity for 9 sessions performed over a 3-week period. As the next step, our system will be expanded with additional competitive, cooperative and single-player games, then tested in full clinical trials in both clinical and home environments.