Situated Dialogue Learning through Procedural Environment Generation (original) (raw)

Supporting Dialogue Generation for Story-Based Games

Providing compelling, realistic, immersive game worlds is one of the major goals in modern game design. The presence of unique and interesting dialogue for all of the characters in a game enhances this sense of immersion. In this paper we present the concept of an "Intentional Dialogue Line" that supports the efficient generation of multiple variations of a dialogue, where these variations are both unique and appropriate to the character who is speaking them. This paper focuses on how machine learning can be used to quickly populate the intentional dialogue lines with existing content.

Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence

arXiv (Cornell University), 2022

AI researchers have posited Dungeons and Dragons (D&D) as a challenge problem to test systems on various language-related capabilities. In this paper, we frame D&D specifically as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history. We create a gameplay dataset consisting of nearly 900 games, with a total of 7,000 players, 800,000 dialogue turns, 500,000 dice rolls, and 58 million words. We automatically annotate the data with partial state information about the game play. We train a large language model (LM) to generate the next game turn, conditioning it on different information. The LM can respond as a particular character or as the player who runs the game-i.e., the Dungeon Master (DM). It is trained to produce dialogue that is either in-character (roleplaying in the fictional world) or out-of-character (discussing rules or strategy). We perform a human evaluation to determine what factors make the generated output plausible and interesting. We further perform an automatic evaluation to determine how well the model can predict the game state given the history and examine how well tracking the game state improves its ability to produce plausible conversational output.

Exploration Based Language Learning for Text-Based Games

ArXiv, 2020

This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents. Moreover, they provide a learning environment in which these skills can be acquired through interactions with an environment rather than using fixed corpora. One aspect that makes these games particularly challenging for learning agents is the combinatorially large action space. Existing methods for solving text-based games are limited to games that are either very simple or have an action space restricted to a predetermined set of admissible actions. In this work, we propose to use the exploration approach of Go-Explore for solving text-based ga...

Automating Direct Speech Variations in Stories and Games

Games and NLP, 2014

Dialogue authoring in large games requires not only content creation but the subtlety of its delivery, which can vary from character to character. Manually authoring this dialogue can be tedious, time-consuming, or even altogether infeasible. This paper utilizes a rich narrative representation for modeling dialogue and an expressive natural language generation engine for realizing it, and expands upon a translation tool that bridges the two. We add functionality to the translator to allow direct speech to be modeled by the narrative representation, whereas the original translator supports only narratives told by a third person narrator. We show that we can perform character substitution in dialogues. We implement and evaluate a potential application to dialogue implementation: generating dialogue for games with big, dynamic, or procedurally-generated open worlds. We present a pilot study on human perceptions of the personalities of characters using direct speech, assuming unknown personality types at the time of authoring.

Algorithmic Improvements for Deep Reinforcement Learning Applied to Interactive Fiction

Proceedings of the AAAI Conference on Artificial Intelligence

Text-based games are a natural challenge domain for deep reinforcement learning algorithms. Their state and action spaces are combinatorially large, their reward function is sparse, and they are partially observable: the agent is informed of the consequences of its actions through textual feedback. In this paper we emphasize this latter point and consider the design of a deep reinforcement learning agent that can play from feedback alone. Our design recognizes and takes advantage of the structural characteristics of text-based games. We first propose a contextualisation mechanism, based on accumulated reward, which simplifies the learning problem and mitigates partial observability. We then study different methods that rely on the notion that most actions are ineffectual in any given situation, following Zahavy et al.'s idea of an admissible action. We evaluate these techniques in a series of text-based games of increasing difficulty based on the TextWorld framework, as well as ...

Learning and Reusing Dialog for Repeated Interactions with a Situated Social Agent

Intelligent Virtual Agents, 2017

Content authoring for conversations is a limiting factor in creating verbal interactions with intelligent virtual agents. Building on techniques utilizing semi-situated learning in an incremental crowdworking pipeline, this paper introduces an embodied agent that self-authors its own dialog for social chat. In particular, the autonomous use of crowdworkers is supplemented with a generalization method that borrows and assesses the validity of dialog across conversational states. We argue that the approach offers a community-focused tailoring of dialog responses that is not available in approaches that rely solely on statistical methods across big data. We demonstrate the advantages that this can bring to interactions through data collected from 486 conversations between a situated social agent and 22 users during a 3 week long evaluation period.

Interactive Narrative Personalization with Deep Reinforcement Learning

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017

Data-driven techniques for interactive narrative generation are the subject of growing interest. Reinforcement learning (RL) offers significant potential for devising data-driven interactive narrative generators that tailor players’ story experiences by inducing policies from player interaction logs. A key open question in RL-based interactive narrative generation is how to model complex player interaction patterns to learn effective policies. In this paper we present a deep RL-based interactive narrative generation framework that leverages synthetic data produced by a bipartite simulated player model. Specifically, the framework involves training a set of Q-networks to control adaptable narrative event sequences with long short-term memory network-based simulated players. We investigate the deep RL framework’s performance with an educational interactive narrative, Crystal Island. Results suggest that the deep RL-based narrative generation framework yields effective personalized int...

How Should Agents Ask Questions For Situated Learning? An Annotated Dialogue Corpus

2021

Intelligent agents that are confronted with novel concepts in situated environments will need to ask their human teammates questions to learn about the physical world. To better understand this problem, we need data about asking questions in situated task-based interactions. To this end, we present the Human-Robot Dialogue Learning (HuRDL) Corpus - a novel dialogue corpus collected in an online interactive virtual environment in which human participants play the role of a robot performing a collaborative tool-organization task. We describe the corpus data and a corresponding annotation scheme to offer insight into the form and content of questions that humans ask to facilitate learning in a situated environment. We provide the corpus as an empirically-grounded resource for improving question generation in situated intelligent agents.

Striving for Author-Friendly Procedural Dialogue Generation

This paper reports on an ongoing attempt to develop an author-friendly approach to procedural game dialogue generation. Various affordances of the experimental authoring tool Expressionist are appropriated to allow non-computer scientist authors to design virtual characters' discourse and reasoning potential. The paper describes how the Hammurabi game project makes use of metadata-driven context free grammars to author virtual characters that can generate not only discourse but also context-relevant decisions. The author-friendliness and generativity of the approach is discussed.

Toward Recombinant Dialogue in Interactive Narrative

Intelligent Narrative Technologies 7, 2014

Prom Week is a social-simulation videogame driven by the artificial intelligence engine Comme il Faut (CiF). In each level of the game, the player selects social interactions between characters in an effort to achieve socially oriented goals. These social interactions are enacted with hand-authored natural-language dialogue exchanges, called instantiations, which also serve to render the underlying social considerations propelling the narrative at hand. While CiF’s merit is in its capacity to richly model a social space, constraints rooted in authorial burden hinder Prom Week’s ability to fully render CiF’s rich social representations. What is needed is more instantiations, specifically instantiations that can render uncommon or complex game states with greater fidelity. We propose a technique to procedurally generate new, felicitous instantiations by recombination of dialogue segments from existing instantiations that are annotated, using the story-encoding tool Scheherazade, for their transmissions about the story world and their various dependencies.