Audition, The Game: Assessing the possibilities of speech as a non-trivial gameplay element in video games (original) (raw)
Related papers
Procedia Computer Science, 2012
This paper describes Audition, The Game, a Flash-based video game created to explore whether video games could contribute to the study and treatment of speech impediments, in particular, stuttering. A pilot study indicated a relation between speech demands, stress and psychophysiological measures. Further research, particularly, with participants who have speech impediments will be required to explore this relationship and the potential role of the game in helping mitigate stuttering incidents.
Talking to Teo: Video game supported speech therapy
Entertainment Computing, 2014
Children with hearing loss diagnosed before they are 2 years old are capable of developing abilities that range from elementary production such as single sounds, or vowels, to those which are more complex, such as spontaneous production of meaningful words and phrases, assuming that they have been treated correctly and opportunely. Speech production skills depend on language and auditory input, and therapy is usually guided by a therapist in personalised sessions. Some children find mechanisation sessions boring or stressful, while others cannot afford such sessions. In contrast, video games have been shown to motivate youngsters. We introduce Talking to Teo, a video game developed and based on verbal therapy and educational objectives, aimed at the rehabilitation of children with early diagnosed hearing disability, and who use aids such as cochlear implants. The software integrates speech recognition for user interaction and benefits from visual feedback. We performed a set of tests with therapists and patients where video game entertainment has evidenced favouring the repetitive approach required during speech mechanisation sessions.
Integrating computer games in speech therapy for children who stutter
In this paper we describe our work with the development of a novel computer game supporting speech therapy for children who stutter. We discuss the motivation for our work, the theoretical background, and outline the plans and strategies for the development process. Finally, we describe a preliminary study carried out to evaluate the potential of integrating computer games in speech therapy for children.
sPeAK-MAN: Towards Popular Gameplay for Speech Therapy
Current speech therapy treatments are not easily accessible to the general public due to cost and demand. Therapy sessions are also laborious and maintaining motivation of patients is hard. We propose using popular games and speech recognition technology for speech therapy in an individualised and accessible manner. sPeAK-MAN is a Pac-Manlike game with a core gameplay mechanic that incorporates vocalisation of words generated from a pool commonly used in clinical speech therapy sessions. Other than improving engagement, sPeAK-MAN aims to provide real-time feedback on the vocalisation performance of patients. It also serves as an initial prototype to demonstrate the possibilities of using familiar popular gameplay (instead of building one from scratch) for rehabilitation purposes.
Combining Think-aloud and Physiological Data to Understand Video Game Experiences
Think-aloud protocols are commonly used to evaluate player experiences of video games but suffer from a lack of objectivity and timeliness. On the other hand, quantitative captures of physiological data are effective; providing detailed, unbiased and continuous responses of players, but lack contexts for interpretation. This paper documents how both approaches could be used together in practice by comparing video-cued retrospective think-aloud data and physiological data collected during a video gameplay experiment. We observed that many interesting physiological responses did not feature in participants' think-aloud data, and conversely, reports of interesting experiences were sometimes not observed in the collected physiological data. Through learnings from our experiment, we present some of the challenges when combining these approaches and offer some guidelines as to how qualitative and quantitative data can be used together to gain deeper insights into player experiences.
Speech-driven mobile games for speech therapy: User experiences and feasibility
International Journal of Speech-Language Pathology, 2018
Purpose: To assist in remote treatment, speech-language pathologists (SLPs) rely on mobile games, which though entertaining, lack feedback mechanisms. Games integrated with automatic speech recognition (ASR) offer a solution where speech productions control gameplay. We therefore performed a feasibility study to assess children's and SLPs' experiences towards speech-controlled games, game feature preferences and ASR accuracy. Method: Ten children with childhood apraxia of speech (CAS), six typically developing (TD) children and seven SLPs trialled five games and answered questionnaires. Researchers also compared the results of ASR to perceptual judgment. Result: Children and SLPs found speech-controlled games interesting and fun, despite ASR-human disagreements. They preferred games with rewards, challenge and multiple difficulty levels. Automatic speech recognition-human agreement was higher for SLPs than children, similar between TD and CAS and unaffected by CAS severity (77% TD, 75% CASincorrect; 51% TD, 47% CAS, 71% SLP-correct). Manual stop recording yielded higher agreement than automatic. Word length did not influence agreement. Conclusion: Children's and SLPs' positive responses towards speech-controlled games suggest that they can engage children in higher intensity practice. Our findings can guide future improvements to the ASR, recording methods and game features to improve the user experience and therapy adherence.
Game Sound Technology …, 2010
means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.
Measuring Performance of Children with Speech and Language Disorders Using A Serious Game
Speech impediment affecting children with hearing loss and speech disorders requires speech therapy and much practice to overcome. To motivate the children to practice more, serious games can be used because children are more inclined to play games. In this paper, we have designed and implemented a serious game that can be used both as therapy and as a tool to measure performance of children with speech impediments.. The game consists of three steps. The first step provides information for parents or therapists to decide if their child needs speech therapy or not. In the second step, the child starts to learn specific words while playing the game. The third step aims to measure the performance of the child and evaluate how much the child has learned at the end of the game. The game has an avatar which can be controlled by the child through speech, with the objective of moving the avatar around the environment to earn coins. The avatar is controlled by both voice commands such as Jump, Ahead, Back, Left, Right and arrow keys of the keyboard. The game allows the child to practice longer hours, compared to clinical approaches under the supervision of a therapist, which are time-limited. Our preliminary performance measurements indicate an improvement of 40% for children who play our game 5 times.
Affective speech elicited with a computer game.
Emotion, 2005
To determine the degree to which emotional changes in speech reflect factors other than arousal, such as valence, the authors used a computer game to induce natural emotional speech. Voice samples were elicited following game events that were either conducive or obstructive to the goal of winning and were accompanied by either pleasant or unpleasant sounds. Acoustic analysis of the speech recordings of 30 adolescents revealed that mean energy, fundamental-frequency level, utterance duration, and the proportion of an utterance that was voiced varied with goal conduciveness; spectral energy distribution depended on manipulations of pleasantness; and pitch dynamics depended on the interaction of pleasantness and goal conduciveness. The results suggest that a single arousal dimension does not adequately characterize a number of emotion-related vocal changes, lending weight to multidimensional theories of emotional response patterning.
User experience research on the affective dimensions of videogame players’ interactions has looked to psychophysiological measures as proxies for emotional states. We add to the growing body of literature on this topic and offer our research as a novel application to the methodological toolbox of sociolinguistics. We present a case study of an interaction occurring in a mixed-gender group (3m/1f) playing a video game that has a potentially sexualized component in gameplay, using a Nintendo Wii-mote joystick as a mimetic penis in the game Mario Party 8. Through a detailed analysis of (1) gameplay interaction, including talk, laughter and gesture, (2) participants’ electrodermal activity (EDA), and (3) a post-game debriefing, we track the varied engagements of the players in-game and with each other, showing how the design of the game itself and the entanglement of hardware and software design with gendered gameplay serve to marginalize and exclude the female participant.