A Survey on Reinforcement Learning for Dialogue Systems (original) (raw)
Related papers
Reinforcement learning for spoken dialogue systems
Proc. NIPS99
Systems in which human users speak to a computer in order to achieve a goal are called spoken dialogue systems. Such systems are some of the few realized examples of open- ended, real-time, goal-oriented interaction between humans and computers, and are therefore an ...
Using reinforcement learning to build a better model of dialogue state
2006
Given the growing complexity of tasks that spoken dialogue systems are trying to handle, Reinforcement Learning (RL) has been increasingly used as a way of automatically learning the best policy for a system to make. While most work has focused on generating better policies for a dialogue manager, very little work has been done in using RL to construct a better dialogue state. This paper presents a RL approach for determining what dialogue features are important to a spoken dialogue tutoring system. Our experiments show that incorporating dialogue factors such as dialogue acts, emotion, repeated concepts and performance play a significant role in tutoring and should be taken into account when designing dialogue systems.
Deep Reinforcement Learning for Dialogue Systems with Dynamic User Goals
2020
Dialogue systems have recently become a widely used system across the world. Some of the functionality offered includes application user interfacing, social conversation, data interaction, and task completion. Most recently, dialogue systems have been developed to autonomously and intelligently interact with users to complete complex tasks in diverse operational spaces. This kind of dialogue system can interact with users to complete tasks such as making a phone call, ordering items online, searching the internet for a question, and more. These systems are typically created by training a machine learning model with example conversational data. One of the existing problems with training these systems is that they require large amounts of realistic user data, which can be challenging to collect and label in large quantities. Our research focuses on modifications to user simulators that "change their mind" mid-episode with the goal of training more robust dialogue agents. We ...
An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System
2002
This paper describes a novel method by which a spoken dialogue system can learn to choose an optimal dialogue strategy from its experience interacting with human users. The method is based on a combination of reinforcement learning and performance modeling of spoken dialogue systems. The reinforcement learning component applies Q-learning Watkins, 1989, while the performance modeling component applies the PARADISE evaluation framework Walker et al., 1997 to learn the performance function reward used in reinforcement learning. We illustrate the method with a spoken dialogue system named elvis EmaiL Voice Interactive System, that supports access to email over the phone. We conduct a set of experiments for training an optimal dialogue strategy on a corpus of 219 dialogues in which human users interact with elvis over the phone. We then test that strategy on a corpus of 18 dialogues. We show that elvis can learn to optimize its strategy selection for agent initiative, for reading messages, and for summarizing email folders.
An ISU dialogue system exhibiting reinforcement learning of dialogue policies
Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Posters & Demonstrations on - EACL '06, 2006
We demonstrate a multimodal dialogue system using reinforcement learning for in-car scenarios, developed at Edinburgh University and Cambridge University for the TALK project 1 . This prototype is the first "Information State Update" (ISU) dialogue system to exhibit reinforcement learning of dialogue strategies, and also has a fragmentary clarification feature. This paper describes the main components and functionality of the system, as well as the purposes and future use of the system, and surveys the research issues involved in its construction. Evaluation of this system (i.e. comparing the baseline system with handcoded vs. learnt dialogue policies) is ongoing, and the demonstration will show both.
Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System
2011
Designing the dialogue policy of a spoken dialogue system involves many nontrivial choices. This paper presents a reinforcement learning approach for automatically optimizing a dialogue policy, which addresses the technical challenges in applying reinforcement learning to a working dialogue system with human users. We report on the design, construction and empirical evaluation of NJFun, an experimental spoken dialogue system that provides users with access to information about fun things to do in New Jersey. Our results show that by optimizing its performance via reinforcement learning, NJFun measurably improves system performance.
2011
This paper describes a novel method by which a spoken dialogue system can learn to choose an optimal dialogue strategy from its experience interacting with human users. The method is based on a combination of reinforcement learning and performance modeling of spoken dialogue systems. The reinforcement learning component applies Q-learning Watkins, 1989, while the performance modeling component applies the PARADISE evaluation framework Walker et al., 1997 to learn the performance function reward used in reinforcement learning. We illustrate the method with a spoken dialogue system named elvis EmaiL Voice Interactive System, that supports access to email over the phone. We conduct a set of experiments for training an optimal dialogue strategy on a corpus of 219 dialogues in which human users interact with elvis over the phone. We then test that strategy on a corpus of 18 dialogues. We show that elvis can learn to optimize its strategy selection for agent initiative, for reading messages, and for summarizing email folders.
Dialogue Systems Domain Interaction Using Reinforcement Learning
2008
This paper describes research about using a reinforcement learning approach to optimize our Domain Knowledge Manager (DKM) that is part of a mixed-initiative task based Spoken Dialogue System (SDS) architecture, namely to access an Ambient Intelligence (AmI) scenario. Assuming that practical dialogue and domain-independent hypothesis are true, we have considered a clear separation between discourse dependent and domain dependent knowledge, which allows reducing the complexity of SDS typical components, specially the Dialoguer Manager (DM). In this context, we believe that is possible to get better DM strategies optimizing the interaction between DM and DKM. For this, we propose a new feature, for the DKM, based on learning and suggest the best task-artifact pairs to satisfy a DM query using the DM feedback as reward. The proposed DKM feature has been tested in our simulator based on Portuguese language.