The Niki and Julie Corpus: Collaborative Multimodal Dialogues between Humans, Robots, and Virtual Agents (original) (raw)
Related papers
How Do People Talk with a Robot? An Analysis of Human-Robot Dialogues in the Real World
This paper reports the preliminary results of a human- robot dialogue analysis in the real world with the goal of understanding users’ interaction patterns. We analyzed the dialogue log data of Roboceptionist, a robotic receptionist located in a high-traffic area in an academic building [2][3]. The results show that (i) the occupation and background (persona) of the robot help people establish common ground with the robot, and (ii) there is great variability in the extent that users follow social norms of human-human dialogues in human-robot dialogues. Based on these results, we describe implications for designing the dialogue of a social robot.
How People Talk to Computers, Robots, and Other Artificial Communication Partners
2006
End-to-end evaluations of more conversational dialogue systems with naive users have uncovered severe usability problems that, among other things, result in low task completion rates. First analyses suggest that these problems are related to the system’s dialogue management and turn-taking behavior. This paper starts with a presentation of experimental results, which shed some light on the effects of that behavior. Based on these findings, some criteria which lie orthogonal to dialogue quality are spelled out. As such, they nevertheless constitute an integral part of a more comprehensive view on dialogue felicity as a function of dialogue quality and efficiency. Since the work on spoken and multimodal dialogue systems presented and discussed herein is aimed at more conversational and adaptive systems, we also show that in certain dialogical situations it is important for such systems to align linguistically towards the users. After describing the corresponding empirical experiments ...
A dialog system for comparative user studies on robot verbal behavior
Robot and Human Interactive …, 2006
In domestic social robot systems the dialog system is often the main user interface. The verbal behavior of such a robot, therefore, plays crucial role in human-robot interaction. Comparative user studies on various verbal behaviors of a robot can effectively contribute to human-robot interaction research. In this paper we present a dialog system that can be easily configured to demonstrate different verbal, initiativetaking behaviors for a robot and, thus, can be used as a platform for such comparative user studies. The pilot study we conducted does not only provide strong evidence for this suitability, but also reveals benefits of comparative studies on a real robot in general.
Speech & Multimodal Resources The Herme Database of Spontaneous Multimodal Human-Robot Dialogues
lrec-conf.org, 2012
This paper presents methodologies and tools for language resource (LR) construction. It describes a database of interactive speech collected over a three-month period at the Science Gallery in Dublin, where visitors could take part in a conversation with a robot. The system collected samples of informal, chatty dialogue -normally difficult to capture under laboratory conditions for human-human dialogue, and particularly so for human-machine interaction. The conversations were based on a script followed by the robot consisting largely of social chat with some task-based elements. The interactions were audio-visually recorded using several cameras together with microphones. As part of the conversation the participants were asked to sign a consent form giving permission to use their data for human-machine interaction research. The multimodal corpus will be made available to interested researchers and the technology developed during the three-month exhibition is being extended for use in education and assisted-living applications.
Engagement During Dialogues with Robots
2005
This paper reports on our research on developing the ability for robots to engage with humans in a collaborative conversation. Engagement is the process by which two (or more) participants establish, maintain and end their perceived connection during interactions they jointly undertake. Many of these interactions are dialogues and we focus on dialogues in which the robot is a host to the human in a physical environment. The paper reports on human-human engagement and its appliction to a robot that collaborates with a human on a demonstration of equipment. AAAI Spring Symposium Dialogical Robots This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment ...
This paper describes a novel experimental setup exploiting state-of-the-art capture equipment to collect a multimodally rich game-solving collaborative multiparty dialogue corpus. The corpus is targeted and designed towards the development of a dialogue system platform to explore verbal and nonverbal tutoring strategies in multiparty spoken interactions. The dialogue task is centered on two participants involved in a dialogue aiming to solve a card-ordering game. The participants were paired into teams based on their degree of extraversion as resulted from a personality test. With the participants sits a tutor that helps them perform the task, organizes and balances their interaction and whose behavior was assessed by the participants after each interaction. Different multimodal signals captured and auto-synchronized by different audio-visual capture technologies, together with manual annotations of the tutor’s behavior constitute the Tutorbot corpus. This corpus is exploited to bui...
The Curious Robot as a Case-Study for Comparing Dialog Systems
Ai Magazine, 2011
Modeling interaction with robots raises new and different challenges for dialog modeling than traditional dialog modeling with less embodied machines. We present four case studies of implementing a typical humanrobot interaction scenario with different state-of-the-art dialog frameworks in order to identify challenges and pitfalls specific to HRI and potential solutions. The results are discussed with a special focus on the interplay between dialog and task modeling on robots.