Expressive Gestures Displayed by a Humanoid Robot during a Storytelling Application (original) (raw)
Related papers
Journal of Visual Language and Computing
The work describes a module that has been implemented for being included in a social humanoid robot architecture, in particular a storyteller robot, named NarRob. This module gives a humanoid robot the capability of mimicking and acquiring the motion of a human user in real-time. This allows the robot to increase the population of his dataset of gestures. The module relies on a Kinect based acquisition setup. The gestures are acquired by observing the typical gesture displayed by humans. The movements are then annotated by several evaluators according to their particular meaning, and they are organized considering a specific typology in the knowledge base of the robot. The properly annotated gestures are then used to enrich the narration of the stories. During the narration, the robot semantically analyses the textual content of the story in order to detect meaningful terms in the sentences and emotions that can be expressed. This analysis drives the choice of the gesture that accompanies the sentences when the story is read.
Design and implementation of an expressive gesture model for a humanoid robot
perso.telecom-paristech.fr
We aim at equipping the humanoid robot NAO with the capacity of performing expressive communicative gestures while telling a story. Given a set of intentions and emotions to convey, our system selects the corresponding gestures from a gestural database, called lexicon. Then it calculates the gestures to be expressive and plans their timing to be synchronized with speech. After that the gestures are instantiated as robot joint values and sent to the robot in order to execute the hand-arm movements. The robot has certain physical constraints to be addressed such as the limits of movement space and joint speed. This article presents our ongoing work on a gesture model generating co-verbal gestures for the robot while taking into account these constraints.
Effects of a robotic storyteller's moody gestures on storytelling perception
2015 International Conference on Affective Computing and Intelligent Interaction (ACII), 2015
A parameterized behavior model was developed for robots to show mood during task execution. In this study, we applied the model to the coverbal gestures of a robotic storyteller. This study investigated whether parameterized mood expression can 1) show mood that is changing over time; 2) reinforce affect communication when other modalities exist; 3) influence the mood induction process of the story; and 4) improve listeners' ratings of the storytelling experience and the robotic storyteller. We modulated the gestures to show either a congruent or an incongruent mood with the story mood. Results show that it is feasible to use parameterized coverbal gestures to express mood evolving over time and that participants can distinguish whether the mood expressed by the gestures is congruent or incongruent with the story mood. In terms of effects on participants we found that mood-modulated gestures (a) influence participants' mood, and (b) influence participants' ratings of the storytelling experience and the robotic storyteller.
Evaluating an Expressive Gesture Model for a Humanoid Robot: Experimental Results
2012
Abstract—This article presents the results obtained from an experiment on our expressive gesture model for humanoid robots. The work took place within a French research project, GVLEX. The goal of this project aims at equipping a humanoid robot with a capacity of producing gestures while telling a story for children. To do that we have extended and developed an existing virtual agent system, named GRETA to adapt to real robots which have certain limits of space and speed movements.
A Verbal and Gestural Corpus of Story Retellings to an Expressive Embodied Virtual Character
We present a corpus of 44 human-agent verbal and gestural story retellings designed to explore whether humans would gesturally entrain to an embodied intelligent virtual agent. We used a novel data collection method where an agent presented story components in installments, which the human would then retell to the agent. At the end of the installments, the human would then retell the embodied animated agent the story as a whole. This method was designed to allow us to observe whether changes in the agent's gestural behavior would result in human gestural changes. The agent modified its gestures over the course of the story, by starting out the first installment with gestural behaviors designed to manifest extraversion, and slowly modifying gestures to express introversion over time, or the reverse. The corpus contains the verbal and gestural transcripts of the human story retellings. The gestures were coded for type, handedness, temporal structure, spatial extent, and the degree to which the participants' gestures match those produced by the agent. The corpus illustrates the variation in expressive behaviors produced by users interacting with embodied virtual characters, and the degree to which their gestures were influenced by the agent's dynamic changes in personality-based expressive style.
Expressive gesture model for humanoid robot
Affective Computing and Intelligent …, 2011
This paper presents an expressive gesture model that generates communicative gestures accompanying speech for the humanoid robot Nao. The research work focuses mainly on the expressivity of robot gestures being coordinated with speech. To reach this objective, we have extended and developed our existing virtual agent platform GRETA to be adapted to the robot. Gestural prototypes are described symbolically and stored in a gestural database, called lexicon. Given a set of intentions and emotional states to communicate the system selects from the robot lexicon corresponding gestures. After that the selected gestures are planned to synchronize speech and then instantiated in robot joint values while taking into account parameters of gestural expressivity such as temporal extension, spatial extension, fluidity, power and repetitivity. In this paper, we will provide a detailed overview of our proposed model.
Multimodal adapted robot behavior synthesis within a narrative human-robot interaction
2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
In human-human interaction, three modalities of communication (i.e., verbal, nonverbal, and paraverbal) are naturally coordinated so as to enhance the meaning of the conveyed message. In this paper, we try to create a similar coordination between these modalities of communication in order to make the robot behave as naturally as possible. The proposed system uses a group of videos in order to elicit specific target emotions in a human user, upon which interactive narratives will start (i.e., interactive discussions between the participant and the robot around each video's content). During each interaction experiment, the humanoid expressive ALICE robot engages and generates an adapted multimodal behavior to the emotional content of the projected video using speech, head-arm metaphoric gestures, and/or facial expressions. The interactive speech of the robot is synthesized using Mary-TTS (text to speech toolkit), which is used-in parallel-to generate adapted head-arm gestures [1]. This synthesized multimodal robot behavior is evaluated by the interacting human at the end of each emotion-eliciting experiment. The obtained results validate the positive effect of the generated robot behavior multimodality on interaction.
Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, 2017
While there has been a growing body of work in childrobot interaction, we still have very little knowledge regarding young children's speaking and listening dynamics and how a robot companion should decode these behaviors and encode its own in a way children can understand. In developing a backchannel prediction model based on observed nonverbal behaviors of 4-6 year-old children, we investigate the effects of an attentive listening robot on a child's storytelling. We provide an extensive analysis of young children's nonverbal behavior with respect to how they encode and decode listener responses and speaker cues. Through a collected video corpus of peer-to-peer storytelling interactions, we identify attention-related listener behaviors as well as speaker cues that prompt opportunities for listener backchannels. Based on our findings, we developed a backchannel opportunity prediction (BOP) model that detects four main speaker cue events based on prosodic features in a child's speech. This rule-based model is capable of accurately predicting backchanneling opportunities in our corpora. We further evaluate this model in a human-subjects experiment where children told stories to an audience of two robots, each with a different backchanneling strategy. We find that our BOP model produces contingent backchannel responses that conveys an increased perception of an attentive listener, and children prefer telling stories to the BOP model robot.
Generation and Evaluation of Communicative Robot Gesture
International Journal of Social Robotics, 2012
How is communicative gesture behavior in robots perceived by humans? Although gesture is crucial in social interaction, this research question is still largely unexplored in the field of social robotics. Thus, the main objective of the present work is to investigate how gestural machine behaviors can be used to design more natural communication in social robots. The chosen approach is twofold. Firstly, the technical challenges encountered when implementing a speech-gesture generation model on a robotic platform are tackled. We present a framework that enables the humanoid robot to flexibly produce synthetic speech and co-verbal hand and arm gestures at run-time, while not being limited to a predefined repertoire of motor actions. Secondly, the achieved flexibility in robot gesture is exploited in controlled experiments. To gain a deeper understanding of how M. Salem ( ) communicative robot gesture might impact and shape human perception and evaluation of human-robot interaction, we conducted a between-subjects experimental study using the humanoid robot in a joint task scenario. We manipulated the non-verbal behaviors of the robot in three experimental conditions, so that it would refer to objects by utilizing either (1) unimodal (i.e., speech only) utterances, (2) congruent multimodal (i.e., semantically matching speech and gesture) or (3) incongruent multimodal (i.e., semantically non-matching speech and gesture) utterances. Our findings reveal that the robot is evaluated more positively when nonverbal behaviors such as hand and arm gestures are displayed along with speech, even if they do not semantically match the spoken utterance.
Integration of gestures and speech in human-robot interaction
2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom), 2012
We present an approach to enhance the interaction abilities of the Nao humanoid robot by extending its communicative behavior with non-verbal gestures (hand and head movements, and gaze following). A set of nonverbal gestures were identified that Nao could use for enhancing its presentation and turn-management capabilities in conversational interactions. We discuss our approach for modeling and synthesizing gestures on the Nao robot. A scheme for system evaluation that compares the values of users' expectations and actual experiences has been presented. We found that open arm gestures, head movements and gaze following could significantly enhance Nao's ability to be expressive and appear lively, and to engage human users in conversational interactions.