The Day a System Becomes a Conversation Partner—Exploring New Horizons in Social Dialogue Systems with Large-scale Deep Learning (original) (raw)

Intelligent Dialogue System Based on Deep Learning Technology

2019

Recent advances in machine learning has contributed to the rebirth of the chat-bot. Lately we have seen a rise in chat-bot technology being made available on the web and on mobile devices, and recent reports states that 57 % of companies have implemented or are planning to implement a chat-bot in the near future. Chat-bots are therefore a big part of an AI powered future, however recent reviews find chat-bots to be perceived as unintelligent and nonconversational. Such findings have not slowed down the rapid implementation of chat-bots online, and the same mistakes seems to be repeated over and over again. This explains why we need to understand how to develop, deploy and monitoring our own dialog system based on "Deep Learning" technologies. In our case studies we have compared different neural network architectures and develop chitchat bot which based on encoder-decoder architecture with attention mechanism. In order to achieve this goal we use Python as programming language, TensorFlow as deep learning framework and GoogleNews word embedding. The peculiarities of the "Deep Learning" technology implementation are discussed in detail. Simulation results confirm the efficiency of the proposed approach for speech recognition.

Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent

2022

We present Chirpy Cardinal, an open-domain social chatbot. Aiming to be both informative and conversational, our bot chats with users in an authentic, emotionally intelligent way. By integrating controlled neural generation with scaffolded, handwritten dialogue, we let both the user and bot take turns driving the conversation, producing an engaging and socially fluent experience. Deployed in the fourth iteration of the Alexa Prize Socialbot Grand Challenge, Chirpy Cardinal handled thousands of conversations per day, placing second out of nine bots with an average user rating of 3.58/5.

DeepPavlov: Open-Source Library for Dialogue Systems

Proceedings of ACL 2018, System Demonstrations, 2018

Adoption of messaging communication and voice assistants has grown rapidly in the last years. This creates a demand for tools that speed up prototyping of featurerich dialogue systems. An open-source library DeepPavlov is tailored for development of conversational agents. The library prioritises efficiency, modularity, and extensibility with the goal to make it easier to develop dialogue systems from scratch and with limited data available. It supports modular as well as end-to-end approaches to implementation of conversational agents. Conversational agent consists of skills and every skill can be decomposed into components. Components are usually models which solve typical NLP tasks such as intent classification, named entity recognition or pre-trained word vectors. Sequence-to-sequence chitchat skill, question answering skill or task-oriented skill can be assembled from components provided in the library.

Developing Dialog Manager in Chatbots via Hybrid Deep Learning Architectures

Advances in Intelligent Systems and Computing

Dialog Manager has played a great role in conversational AI so much, so that it is also called the heart of a dialog system. It has been employed in task-oriented Chatbot to learn the context of a conversation and come up with some representation which helps in executing the task. For example, booking a restaurant table, flight booking, movie tickets, etc. In this paper, a dialog manager is trained in a supervised manner in order to predict the best response given the latent state representation of the user message. The latent representation is formed by the Convolution Neural Network (CNN) and Bidirectional Long Short Term Memory network (BiLSTM) with attention. An ablation study is conducted with three different architectures. One of them achieved a state-of-the-art result in turn accuracy on babI6 dataset and dialog accuracy equivalent to the baseline model. Keywords Word2vec (w2v) • Long Short Term Memory(LSTM) • Convolution Neural Network (CNN) • Bag of Words (BoW) • BiLSTM • One dimensional convolution neural network (1DCNN)

Data-Driven Dialogue Systems for Social Agents

In order to build dialogue systems to tackle the ambitious task of holding social conversations, we argue that we need a data-driven approach that includes insight into human conversational "chit-chat", and which incorporates different natural language processing modules. Our strategy is to analyze and index large corpora of social media data, including Twitter conversations, online debates, dialogues between friends, and blog posts, and then to couple this data retrieval with modules that perform tasks such as sentiment and style analysis, topic modeling, and summarization. We aim for personal assistants that can learn more nuanced human language, and to grow from task-oriented agents to more personable social bots.

Generative Deep Neural Networks for Dialogue: A Short Review

Researchers have recently started investigating deep neural networks for dialogue applications. In particular, generative sequence-to-sequence (Seq2Seq) models have shown promising results for unstructured tasks, such as word-level dialogue response generation. The hope is that such models will be able to leverage massive amounts of data to learn meaningful natural language representations and response generation strategies, while requiring a minimum amount of domain knowledge and hand-crafting. An important challenge is to develop models that can effectively incorporate dialogue context and generate meaningful and diverse responses. In support of this goal, we review recently proposed models based on generative encoder-decoder neural network architectures, and show that these models have better ability to incorporate long-term dialogue history, to model uncertainty and ambiguity in dialogue, and to generate responses with high-level compositional structure.

A Knowledge-Grounded Neural Conversation Model

Neural network models are capable of generating extremely natural sounding conversational interactions. However, these models have been mostly applied to casual scenarios (e.g., as "chatbots") and have yet to demonstrate they can serve in more useful conversational applications. This paper presents a novel, fully data-driven, and knowledge-grounded neural conversation model aimed at producing more contentful responses. We generalize the widely-used Sequence-to-Sequence (SEQ2SEQ) approach by conditioning responses on both conversation history and external "facts", allowing the model to be versatile and applicable in an open-domain setting. Our approach yields significant improvements over a competitive SEQ2SEQ baseline. Human judges found that our outputs are significantly more informative.

Further Advances in Open Domain Dialog Systems in the Third Alexa Prize Socialbot Grand Challenge

2020

Building open domain conversational systems that allow users to have engaging conversations on topics of their choice is a challenging task. The Alexa Prize Socialbot Grand Challenge was launched in 2016 to tackle the problem of achieving natural, sustained, coherent and engaging open-domain dialogs. In the third iteration of the competition, university teams have moved the needle on the state of the art, bringing together common sense knowledge representations, neural response generation models, NLU systems enhanced by large-scale transformer models and improved dialog policies to switch between graph-based representations or retrieval-based or templated dialog fragments, along with generated responses. The Third Socialbot Grand Challenge included an improved version of the CoBot (conversational bot) toolkit from the prior competition, along with topic and dialog act detection models, conversation evaluators, and a sensitive content detection model so that the competing teams could...

A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots

arXiv (Cornell University), 2021

In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. Dialogue systems are increasingly being designed to move beyond just imitating conversation and also improve from such interactions over time. In this survey, we present a broad overview of methods developed to build dialogue systems over the years. Different use cases for dialogue systems ranging from task-based systems to open domain chatbots motivate and necessitate specific systems. Starting from simple rule-based systems, research has progressed towards increasingly complex architectures trained on a massive corpus of datasets, like deep learning systems. Motivated with the intuition of resembling human dialogues, progress has been made towards incorporating emotions into the natural language generator, using reinforcement learning. While we see a trend of highly marginal improvement on some metrics, we find that limited justification exists for the metrics, and evaluation practices are not uniform. To conclude, we flag these concerns and highlight possible research directions.

Towards Deep Conversational Recommendations

ArXiv, 2018

There has been growing interest in using neural networks and deep learning techniques to create dialogue systems. Conversational recommendation is an interesting setting for the scientific exploration of dialogue with natural language as the associated discourse involves goal-driven dialogue that often transforms naturally into more free-form chat. This paper provides two contributions. First, until now there has been no publicly available large-scale data set consisting of real-world dialogues centered around recommendations. To address this issue and to facilitate our exploration here, we have collected ReDial, a data set consisting of over 10,000 conversations centered around the theme of providing movie recommendations. We make this data available to the community for further research. Second, we use this dataset to explore multiple facets of conversational recommendations. In particular we explore new neural architectures, mechanisms and methods suitable for composing conversat...