Jelte van Waterschoot - Academia.edu (original) (raw)

Uploads

Papers by Jelte van Waterschoot

Proceedings of the 18th ACM International Conference on Multimodal Interaction, 2016

We present a demonstration of the ARIA framework, a modular approach for rapid development of vir... more We present a demonstration of the ARIA framework, a modular approach for rapid development of virtual humans for information retrieval that have linguistic, emotional, and social skills and a strong personality. We demonstrate the framework's capabilities in a scenario where 'Alice in Wonderland', a popular English literature book, is embodied by a virtual human representing Alice. The user can engage in an information exchange dialogue, where Alice acts as the expert on the book, and the user as an interested novice. Besides speech recognition, sophisticated audiovisual behaviour analysis is used to inform the core agent dialogue module about the user's state and intentions, so that it can go beyond simple chat-bot dialogue. The behaviour generation module features a unique new capability of being able to deal gracefully with interruptions of the agent. CCS Concepts •Human-centered computing → Interaction devices; •Computing methodologies → Machine learning; Activity recognition and understanding;

We reflect on user privacy concerns, transparency and informed consent for long-term interactions... more We reflect on user privacy concerns, transparency and informed consent for long-term interactions with personalized conversational agents. We argue that the common practice of asking users to sign an informed consent form is insufficient to accommodate the privacy concerns of the user. We propose that long-term engaging personalized conversational agents must include an explicit mechanism in their conversations to allow users to have control over their personal information and to have transparency, i.e. about what is stored and who is allowed to view the stored personal information.

ArXiv, 2017

By utilizing different communication channels, such as verbal language, gestures or facial expres... more By utilizing different communication channels, such as verbal language, gestures or facial expressions, virtually embodied interactive humans hold a unique potential to bridge the gap between human-computer interaction and actual interhuman communication. The use of virtual humans is consequently becoming increasingly popular in a wide range of areas where such a natural communication might be beneficial, including entertainment, education, mental health research and beyond. Behind this development lies a series of technological advances in a multitude of disciplines, most notably natural language processing, computer vision, and speech synthesis. In this paper we discuss a Virtual Human Journalist, a project employing a number of novel solutions from these disciplines with the goal to demonstrate their viability by producing a humanoid conversational agent capable of naturally eliciting and reacting to information from a human user. A set of qualitative and quantitative evaluation ...

An important objective in health-technology is the ability to gather information about people’s w... more An important objective in health-technology is the ability to gather information about people’s well-being. Structured interviews can be used to obtain this information, but are time-consuming and not scalable. Questionnaires provide an alternative way to extract such information, though typically lack depth. In this paper, we present our first prototype of the BLISS agent, an artificial intelligent agent which intends to automatically discover what makes people happy and healthy. The goal of Behaviour-based Language-Interactive Speaking Systems (BLISS) is to understand the motivations behind people’s happiness by conducting a personalized spoken dialogue based on a happiness model. We built our first prototype of the model to collect 55 spoken dialogues, in which the BLISS agent asked questions to users about their happiness and well-being. Apart from a description of the BLISS architecture, we also provide details about our dataset, which contains over 120 activities and 100 motiv...

Proceedings of the 18th International Conference on Intelligent Virtual Agents - IVA '18, 2018

We present a new dialogue engine called Flipper 2.0 (Flipper) which aims to help developers of em... more We present a new dialogue engine called Flipper 2.0 (Flipper) which aims to help developers of embodied conversational agents (ECAs) to quickly and flexibly create dialogues. Flipper provides a technically stable and robust dialogue management system to integrate with other components of ECAs such as behaviour realisers. We compare Flipper with state-of-the-art dialogue design systems. We describe the details of our dialogue engine, how it handles dialogue management and how it supports the authoring of dialogues. We demonstrate the use of the dialogue engine with examples of design patterns and discuss practical applications. Finally we give recommendations on the cases in which it is beneficial to use Flipper. CCS CONCEPTS • Human-centered computing → Natural language interfaces; Systems and tools for interaction design; User interface toolkits; • Information systems → Open source software;

The contribution of this paper is twofold: 1) we provide a public corpus for Human-Agent Interact... more The contribution of this paper is twofold: 1) we provide a public corpus for Human-Agent Interaction (where the agent is controlled by a Wizard of Oz) and 2) we show a study on verbal alignment in Human-Agent Interaction, to exemplify the corpus' use. In our recordings for the Human-Agent Interaction Alice-corpus (HAI Alice-corpus), participants talked to a wizarded agent, who provided them with information about the book Alice in Wonderland and its author. The wizard had immediate and almost full control over the agent's verbal and nonverbal behavior, as the wizard provided the agent's speech through his own voice and his facial expressions were directly copied onto the agent. The agent's hand gestures were controlled through a button interface. Data was collected to create a corpus with unexpected situations, such as misunderstandings, (accidental) false information, and interruptions. The HAI Alice-corpus consists of transcribed audio-video recordings of 15 conver...

In this paper we discuss our road map to design a Dutch spoken conversational agent that helps ol... more In this paper we discuss our road map to design a Dutch spoken conversational agent that helps older adults with self-management of their well-being. We conducted three usability studies with our conversational agent, each time after discussing it with different stakeholders. We discuss the challenges incorporating each stakeholder’s wishes and needs whilst iteratively designing our agent.

In SAIBA-compliant agent systems, the Function Markup Language (FML) is used to describe the agen... more In SAIBA-compliant agent systems, the Function Markup Language (FML) is used to describe the agent’s communicative functions that are transformed into utterances accompanied with appropriate non-verbal behaviours. In the context of the ARIA Framework, we propose a template-based approach, grounded in the DIT++ taxonomy, as an interface between the dialogue manager (DM) and the non-verbal behaviour generation (NVBG) components of this framework. Our approach enhances our current FML-APML implementation of FML with the capability of receiving on-the-fly generated natural language and socio-emotional parameters (e.g. emotional stance) for transforming the agent’s intents in believable verbal and non-verbal behaviours in an adaptive manner.

Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization

Proceedings of the 18th ACM International Conference on Multimodal Interaction, 2016

We present a demonstration of the ARIA framework, a modular approach for rapid development of vir... more We present a demonstration of the ARIA framework, a modular approach for rapid development of virtual humans for information retrieval that have linguistic, emotional, and social skills and a strong personality. We demonstrate the framework's capabilities in a scenario where 'Alice in Wonderland', a popular English literature book, is embodied by a virtual human representing Alice. The user can engage in an information exchange dialogue, where Alice acts as the expert on the book, and the user as an interested novice. Besides speech recognition, sophisticated audiovisual behaviour analysis is used to inform the core agent dialogue module about the user's state and intentions, so that it can go beyond simple chat-bot dialogue. The behaviour generation module features a unique new capability of being able to deal gracefully with interruptions of the agent. CCS Concepts •Human-centered computing → Interaction devices; •Computing methodologies → Machine learning; Activity recognition and understanding;

We reflect on user privacy concerns, transparency and informed consent for long-term interactions... more We reflect on user privacy concerns, transparency and informed consent for long-term interactions with personalized conversational agents. We argue that the common practice of asking users to sign an informed consent form is insufficient to accommodate the privacy concerns of the user. We propose that long-term engaging personalized conversational agents must include an explicit mechanism in their conversations to allow users to have control over their personal information and to have transparency, i.e. about what is stored and who is allowed to view the stored personal information.

ArXiv, 2017

By utilizing different communication channels, such as verbal language, gestures or facial expres... more By utilizing different communication channels, such as verbal language, gestures or facial expressions, virtually embodied interactive humans hold a unique potential to bridge the gap between human-computer interaction and actual interhuman communication. The use of virtual humans is consequently becoming increasingly popular in a wide range of areas where such a natural communication might be beneficial, including entertainment, education, mental health research and beyond. Behind this development lies a series of technological advances in a multitude of disciplines, most notably natural language processing, computer vision, and speech synthesis. In this paper we discuss a Virtual Human Journalist, a project employing a number of novel solutions from these disciplines with the goal to demonstrate their viability by producing a humanoid conversational agent capable of naturally eliciting and reacting to information from a human user. A set of qualitative and quantitative evaluation ...

An important objective in health-technology is the ability to gather information about people’s w... more An important objective in health-technology is the ability to gather information about people’s well-being. Structured interviews can be used to obtain this information, but are time-consuming and not scalable. Questionnaires provide an alternative way to extract such information, though typically lack depth. In this paper, we present our first prototype of the BLISS agent, an artificial intelligent agent which intends to automatically discover what makes people happy and healthy. The goal of Behaviour-based Language-Interactive Speaking Systems (BLISS) is to understand the motivations behind people’s happiness by conducting a personalized spoken dialogue based on a happiness model. We built our first prototype of the model to collect 55 spoken dialogues, in which the BLISS agent asked questions to users about their happiness and well-being. Apart from a description of the BLISS architecture, we also provide details about our dataset, which contains over 120 activities and 100 motiv...

Proceedings of the 18th International Conference on Intelligent Virtual Agents - IVA '18, 2018

We present a new dialogue engine called Flipper 2.0 (Flipper) which aims to help developers of em... more We present a new dialogue engine called Flipper 2.0 (Flipper) which aims to help developers of embodied conversational agents (ECAs) to quickly and flexibly create dialogues. Flipper provides a technically stable and robust dialogue management system to integrate with other components of ECAs such as behaviour realisers. We compare Flipper with state-of-the-art dialogue design systems. We describe the details of our dialogue engine, how it handles dialogue management and how it supports the authoring of dialogues. We demonstrate the use of the dialogue engine with examples of design patterns and discuss practical applications. Finally we give recommendations on the cases in which it is beneficial to use Flipper. CCS CONCEPTS • Human-centered computing → Natural language interfaces; Systems and tools for interaction design; User interface toolkits; • Information systems → Open source software;

The contribution of this paper is twofold: 1) we provide a public corpus for Human-Agent Interact... more The contribution of this paper is twofold: 1) we provide a public corpus for Human-Agent Interaction (where the agent is controlled by a Wizard of Oz) and 2) we show a study on verbal alignment in Human-Agent Interaction, to exemplify the corpus' use. In our recordings for the Human-Agent Interaction Alice-corpus (HAI Alice-corpus), participants talked to a wizarded agent, who provided them with information about the book Alice in Wonderland and its author. The wizard had immediate and almost full control over the agent's verbal and nonverbal behavior, as the wizard provided the agent's speech through his own voice and his facial expressions were directly copied onto the agent. The agent's hand gestures were controlled through a button interface. Data was collected to create a corpus with unexpected situations, such as misunderstandings, (accidental) false information, and interruptions. The HAI Alice-corpus consists of transcribed audio-video recordings of 15 conver...

In this paper we discuss our road map to design a Dutch spoken conversational agent that helps ol... more In this paper we discuss our road map to design a Dutch spoken conversational agent that helps older adults with self-management of their well-being. We conducted three usability studies with our conversational agent, each time after discussing it with different stakeholders. We discuss the challenges incorporating each stakeholder’s wishes and needs whilst iteratively designing our agent.

In SAIBA-compliant agent systems, the Function Markup Language (FML) is used to describe the agen... more In SAIBA-compliant agent systems, the Function Markup Language (FML) is used to describe the agent’s communicative functions that are transformed into utterances accompanied with appropriate non-verbal behaviours. In the context of the ARIA Framework, we propose a template-based approach, grounded in the DIT++ taxonomy, as an interface between the dialogue manager (DM) and the non-verbal behaviour generation (NVBG) components of this framework. Our approach enhances our current FML-APML implementation of FML with the capability of receiving on-the-fly generated natural language and socio-emotional parameters (e.g. emotional stance) for transforming the agent’s intents in believable verbal and non-verbal behaviours in an adaptive manner.

Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization