Embodied conversational characters: representation formats for multimodal communicative behaviours (original) (raw)

Towards a common framework for multimodal generation: The behavior markup language

Intelligent Virtual …, 2006

This paper describes an international effort to unify a multimodal behavior generation framework for Embodied Conversational Agents (ECAs). We propose a three stage model we call SAIBA where the stages represent intent planning, behavior planning and behavior realization. A Function Markup Language (FML), describing intent without referring to physical behavior, mediates between the first two stages and a Behavior Markup Language (BML) describing desired physical realization, mediates between the last two stages. In this paper we will focus on BML.

MURML: A Multimodal Utterance Representation Markup Language for Conversational Agents

2002

This paper presents work on an artificial anthropomorphic agent with multimodal interaction abilitities. It focuses on the development of a markup language, MURML, that bridges between the planning and the animation tasks in the production of multimodal utterances. This hierarchically structured notation provides flexible means of describing gestures in a form-based way and of explicitly experessing their relations to accompanying speech.

Representation of expressivity for embodied conversational agents

… on Autonomous Agents & Multi-Agent …, 2004

We aim at creating an Embodied Conversational Agent (ECA) that would exhibit not only a consistent behavior with her personality and contextual environment factors but also that would be defined as an individual and not as a generic agent. The behavior of an agent depends not only on factors defining her individuality (such as her culture, her social and professional role, her personality and her experience), but also on a set of contextual (such as her interlocutor, the social conversation setting), and dynamic variables (belief, goal, emotion). We call these types of factors 'influences' in the sense that they affect the behaviors to be displayed. As humans vary greatly in their way in expressing a given meaning, there exist many manners to map a meaning into a set of signals. Moreover influences may act at different levels: they may act not only on the selection of a non-verbal behavior to convey a meaning but also on its expressivity. In this paper we present how we model influences working on the signals. We also describe our computational model of the agent's expressivity.

Multimodal expressive embodied conversational agents

2005

Abstract In this paper we present our work toward the creation of a multimodal expressive Embodied Conversational Agent (ECA). Our agent, called Greta, exhibits nonverbal behaviors synchronized with speech. We are using the taxonomy of communicative functions developed by Isabella Poggi [22] to specify the behavior of the agent. Based on this taxonomy a representation language, Affective Presentation Markup Language, APML has been defined to drive the animation of the agent [4].

Realizing Multimodal Behavior Closing the gap between behavior planning and embodied agent presentation

2011

Abstract. Generating coordinated multimodal behavior for an embodied agent (speech, gesture, facial expression...) is challenging. It requires a high degree of animation control, in particular when reactive behaviors are required. We suggest to distinguish realization planning, where gesture and speech are processed symbolically using the behavior markup language (BML), and presentation which is controlled by a lower level animation language (EMBRScript). Reactive behaviors can bypass planning and directly control presentation. In this paper, we show how to define a behavior lexicon, how this lexicon relates to BML and how to resolve timing using formal constraint solvers. We conclude by demonstrating how to integrate reactive emotional behaviors. 1

Actions speak louder than words: An intentional look at defining communicative acts for embodied conversational agents

2009

This paper takes a philosophical look at how one might use insights from examining the nature of action in order to help in defining a set of multi-modal communicative actions for use in and by embodied conversational agents. A key theme of the paper is the central importance of intention in communicative actions. The paper also offers some suggestions on how to conceive of and categorise different types of action (communicative and non-communicative) so as to set a firm foundation before tackling the problem of defining a set of communicative actions for embodied conversational agents. It also gives some more specific practical advice on how to begin going about defining that set.

Representation and Reasoning in a Multimodal Conversational Character

2001

Abstract We describe the reasoning mechanisms used in a fully-implemented dialogue system. This dialogue system, based on a speech acts formalism, supports a multimodal conversational character for Interactive Television. The system maintains an explicit representation of programme descriptions, which also constitutes an attentional structure.

04121 Abstracts Collection -- Evaluating Embodied Conversational Agents

Dagstuhl Seminars, 2004

From 14.03.04 to 19.03.04, the Dagstuhl Seminar 04121 Evaluating Embodied Conversational Agents was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The rst section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available.

Algorithms for controlling cooperation between output modalities in 2D embodied conversational agents

Proceedings of the 5th international conference on Multimodal interfaces - ICMI '03, 2003

Recent advances in the specification of the multimodal behavior of Embodied Conversational Agents (ECA) have proposed a direct and deterministic one-step mapping from high-level specifications of dialog state or agent emotion onto low-level specifications of the multimodal behavior to be displayed by the agent (e.g. facial expression, gestures, vocal utterance). The difference of abstraction between these two levels of specification makes difficult the definition of such a complex mapping. In this paper we propose an intermediate level of specification based on combinations between modalities (e.g. redundancy, complementarity). We explain how such intermediate level specifications can be described using XML in the case of deictic expressions. We define algorithms for parsing such descriptions and generating the corresponding multimodal behavior of 2D cartoon-like conversational agents. Some random selection has been introduced in these algorithms in order to induce some "natural variations" in the agent's behavior. We conclude on the usefulness of this approach for the design of ECA.

Evaluation of Multimodal Behaviour of Embodied Agents

Human-Computer Interaction Series, 2004

Individuality of Embodied Conversational Agents (ECAs) may depend on both the look of the agent and the way it combines different modalities such as speech and gesture. In this chapter, we describe a study in which male and female users had to listen to three short technical presentations made by ECAs. Three multimodal strategies of ECAs for using arm gestures with speech were compared: redundancy, complementarity, and speech-specialization. These strategies were randomly attributed to different-looking 2D ECAs, in order to test independently the effects of multimodal strategy and ECA's appearance. The variables we examined were subjective impressions and recall performance. Multimodal strategies proved to influence subjective ratings of quality of explanation, in particular for male users. On the other hand, appearance affected likeability, but also recall performance. These results stress the importance of both multimodal strategy and appearance to ensure pleasantness and effectiveness of presentation ECAs.