James Ryan - Academia.edu (original) (raw)

Papers by James Ryan

CHI Conference on Human Factors in Computing Systems, 2020

Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove diffi... more Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial pilot study comparing it with traditional web-based educational materials from an existing RCR course. The preliminary results highlight that utilizing a choice-based interactive story game may prove more effective for RCR education, with significantly higher engagement and comparable or better scores for tests of RCR topics.

2nd Workshop on Natural Language Processing for Conversational AI, 2020

Dialog State Tracking (DST) is a problem space in which the effective vocabulary is practically l... more Dialog State Tracking (DST) is a problem space in which the effective vocabulary is practically limitless. For example, the domain of possible movie titles or restaurant names is bound only by the limits of language. As such, DST systems often encounter out-of-vocabulary words at inference time that were never encountered during training. To combat this issue, we present a targeted data augmentation process, by which a practitioner observes the types of errors made on held-out evaluation data, and then modifies the training data with additional corpora to increase the vocabulary size at training time. Using this with a RoBERTa-based Transformer architecture, we achieve state-of-the-art results in comparison to systems that only mask trouble slots with special tokens. Additionally, we present a data-representation scheme for seamlessly retarget-ing DST architectures to new domains.

International Conference on the Foundations of Digital Games, 2020

Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove diffi... more Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial study comparing it with traditional web-based educational materials from an existing university RCR course. The results highlight that utilizing a choice-based interactive story game is more effective for RCR education, with learners developing significantly higher engagement, stronger overall moral reasoning skills, and better knowledge scores for certain RCR topics.

PhD Thesis, University of California, Santa Cruz, 2018

There is a peculiar method in the area of procedural narrative called emergent narrative: instead... more There is a peculiar method in the area of procedural narrative called emergent narrative: instead of automatically inventing stories or deploying authored narrative content, a system simulates a storyworld out of which narrative may emerge from the happenstance of character activity in that world. It is the approach taken by some of the most successful works in the history of computational media (The Sims, Dwarf Fortress), but curiously also some of its most famous failures (Sheldon Klein's automatic novel writer, Tale-Spin). How has this been the case? To understand the successes, we might ask this essential question: what is the pleasure of emergent narrative? I contend that the form works more like nonfiction than fiction---emergent stories actually happen---and this produces a peculiar aesthetics that undergirds the appeal of its successful works. What then is the pain of emergent narrative? There is a ubiquitous tendency to misconstrue the raw transpiring of a simulation (or a trace of that unfolding) as being a narrative artifact, but such material will almost always lack story structure. So, how can the pain of emergent narrative be alleviated while simultaneously maintaining the pleasure? This dissertation introduces a refined approach to the form, called curationist emergent narrative (or just curationism), that aims to provide an answer to this question. Instead of treating the raw material of simulation as a story, in curationism that material is curated to construct an actual narrative artifact that is then mounted in a full-fledged media experience (to enable human encounter with the artifact). This recasts story generation as an act of recounting, rather than invention. I believe that curationism can also explain how both wild successes and phenomenal failures have entered the oeuvre of emergent narrative: in successful works, humans have taken on the burden of curating an ongoing simulation to construct a storied understanding of what has happened, while in the failures humans have not been willing to do the necessary curation. Without curation, actual stories cannot obtain in emergent narrative. But what if a storyworld could curate itself? That is, can we build systems that automatically recount what has happened in simulated worlds? In the second half of this dissertation, I provide an autoethnography and a collection of case studies that recount my own personal (and collaborative) exploration of automatic curation over the course of the last six years. Here, I report the technical, intellectual, and media-centric contributions made by three simulation engines (World, Talk of the Town, Hennepin) and three second-order media experiences that are respectively driven by those engines (Diol/Diel/Dial, Bad News, Sheldon County). In total, this dissertation provides a loose history of emergent narrative, an apologetics of the form, a polemic against it, a holistic refinement (maintaining the pleasure while killing the pain), and reports on a series of artifacts that represent a gradual instantiation of that refinement. To my knowledge, this is the most extensive treatment of emergent narrative to yet appear.

10th International Conference on Interactive Digital Storytelling, 2017

The first meeting of a new workshop on the History of Expressive Systems (HEX) is being held at I... more The first meeting of a new workshop on the History of Expressive Systems (HEX) is being held at ICIDS 2017. By 'expressive systems', we broadly mean computer systems (or predigital procedural methods) that were developed with expressive or creative aims; this is meant to be a superset of the areas called creative AI, expressive AI, videogame AI, computational creativity, interactive storytelling, computational narrative, procedural music, computer poetry, generative art, and more. While much of this purview intersects with projects in artificial intelligence , we are more broadly interested in procedural methods of all kinds (even predigital ones, as mentioned above). HEX is meant to illuminate and celebrate the history of systems in this area, especially the untold histories of projects that are today forgotten or relatively unknown.

10th International Conference on Interactive Digital Storytelling, 2017

We provide the first extensive account of an unknown story generator that was developed by lingui... more We provide the first extensive account of an unknown story generator that was developed by linguist Joseph E. Grimes in the early 1960s. A pioneering system, it was the first to take a grammar-based approach and the first to operationalize Propp's famous model. This is the opening paper in a series that will aim to reformulate the prevailing history of story generation in light of new findings we have made pertaining to several forgotten early projects. Our study here has been made possible by personal communication with the system's creator, Grimes, and excavation of three obscure contemporaneous sources. While the accepted knowledge in our field is that the earliest story generator was Sheldon Klein's automatic novel writer, first reported in 1971, we show that Grimes's system and two others preceded it. In doing this, we reveal a new earliest known system. With this paper, and follow-ups to it that are in progress, we aim to provide a new account of the area of story generation that lends our community insight as to where it came from and where it should go next. We hope others will join us in this mission.

This document presents a translation into English, by Rogelio E. Cardona-Rivera, of the section t... more This document presents a translation into English, by Rogelio E. Cardona-Rivera, of the section titled 'La Simulación' in Joseph E. Grimes's article "La computadora en las investigaciones humanísticas", which appeared in the Spanish-language journal Anuario de Letras. Lingística y Filología in 1965. That section is the most extensive contemporaneous account by Grimes of his exploration of computer story generation in the early 1960s, which produced one of the earliest known systems in that area. The source text was translated sentence by sentence, and its text stylization, parenthetical elements, and footnote are preserved; the bibliography is an artifact of this document, not the original one being translated. James Ryan rediscovered the original article, arranged for its translation, and prepared this document.

We extend the Expressionist project, and thereby the re-emerging area of grammar-based text gener... more We extend the Expressionist project, and thereby the re-emerging area of grammar-based text generation, by applying a technique from software verification to a critical search problem related to content generation from grammars. In Expressionist, authors attach tags (corresponding to pertinent meanings) to nonterminal symbols in a context-free grammar , which enables the targeted generation of content that expresses requested meanings (i.e., has the requested tags). While previous work has demonstrated methods for requesting content with a single required tag, requests for multiple tags yields a search task over domains that may realistically span quintillions or more elements. In this paper, we reduce Expressionist grammars to symbolic visibly pushdown automata, which allows us to locate in massive search spaces generable outputs that satisfy moderately complex criteria related to tags. While the satisficing of more complex tag criteria is still not feasible using this technique, we forecast a number of opportunities for future directions.

The AIIDE Playable Experiences track celebrates innovations in how AI can be used in polished int... more The AIIDE Playable Experiences track celebrates innovations in how AI can be used in polished interactive experiences. Four 2016 accepted submissions display a diversity of approaches. Rogue Process combines techniques for medium-permanence procedurally generated hacking worlds. Elsinore applies temporal predicate logic to enable a time-traveling narrative with character simulation. A novel level generator uses conceptual blending to translate Mario Bros. design styles across levels. And Bad News uses deep simulation of a town and it's residents to ground a mixed-reality performance. Together these playable experiences showcase the opportunities for AI in interactive experiences. AIIDE playable experiences submissions showcase innovations in using AI to create interactive experiences. Traditionally AI is a secondary consideration, used to support an existing interactive experience. Yet AI can enable novel interactive experiences using existing AI techniques and even foster innovations in AI techniques to produce new kinds of playable experiences. The playable experiences track celebrates these efforts and emphasizes the development of polished experiences that can be played by novice users— demonstrating the potential for AI to reach a broad audience. The 2016 AIIDE Playable Experiences track (chaired by Alex Zook) includes four entries that demonstrate a breadth of experience domains and AI techniques. AI can provide a unique setting for interaction: Bad News simulates the history of a small town, down to the level of characters, to seed a live performance; Rogue Process generates fictional corporations and their skyscraper offices; and a conceptual blending approach learns about Mario Bros. level designs from data and applies design motifs to new levels. Alternatively, AI can drive the experience itself: Elsinore simulates char

AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Oct 10, 2016

The Expressive Intelligence Studio is developing a new approach to freeform conversational intera... more The Expressive Intelligence Studio is developing a new approach to freeform conversational interaction in playable media that combines dialogue management, natural language generation (NLG), and natural language understanding. In this paper, we present our method for dialogue generation, which has been fully implemented in a game we are developing called Talk of the Town. Eschewing a traditional NLG pipeline, we take up a novel approach that combines human language expertise with computer generativity. Specifically, this method utilizes a tool that we have developed for authoring context-free grammars (CFGs) whose productions come packaged with explicit metadata. Instead of terminally expanding top-level symbols—the conventional way of generating from a CFG—we employ an unusual middle-out procedure that targets mid-level symbols and traverses the grammar by both forward chaining and backward chaining, expanding symbols conditionally by testing against the current game state. In this paper, we present our method, discuss a series of associated authoring patterns, and situate our approach against the few earlier projects in this area.

Intelligent Virtual Agents, Sep 2016

University of California, Santa Cruz, Technical Report UCSC-SOE-16-11, Jul 22, 2016

In this paper, we present a novel approach to natural language understanding that utilizes contex... more In this paper, we present a novel approach to natural language understanding that utilizes context-free grammars (CFGs) in conjunction with sequence-to-sequence (seq2seq) deep learning. Specifically, we take a CFG authored to generate dialogue for our target application for NLU, a videogame, and train a long short-term memory (LSTM) recurrent neural network (RNN) to map the surface utterances that it produces to traces of the grammatical expansions that yielded them. Critically, this CFG was authored using a tool we have developed that supports arbitrary annotation of the nonterminal symbols in the grammar. Because we already annotated the symbols in this grammar for the semantic and pragmatic considerations that our game's dialogue manager operates over, we can use the grammatical trace associated with any surface utterance to infer such information. During gameplay, we translate player utterances into grammatical traces (using our RNN), collect the markup attributed to the symbols included in that trace, and pass this information to the dialogue manager, which updates the conversation state accordingly. From an offline evaluation task, we demonstrate that our trained RNN translates surface utterances to grammatical traces with great accuracy. To our knowledge, this is the first usage of seq2seq learning for conversational agents (our game's characters) who explicitly reason over semantic and pragmatic considerations.

International Conference on Interactive Digital Storytelling, Nov 2016

Dreams of the prospect of computational narrative suggest a future of deeply interactive, generat... more Dreams of the prospect of computational narrative suggest a future of deeply interactive, generative, and personalized fictional experiences that engage our empathy, curiosity, and sense of responsibility. But the gulf between our current moment and that future is vast. How do we begin to bridge that divide now, both for learning more specifics of these potentials and to create experiences today that can have some of their impact on audiences? We present Bad News, which uses a combination of theatrical performance practices, computational support, and Wizard-of-Oz interaction techniques. Together, these allow for rich, real-time interaction with a procedurally generated story world. We believe our approach is one that could enable other research groups to explore similar territory—and that the resulting experience is engaging and affecting in ways that help strengthen the case for our envisioned futures and also makes the case for trying to field such experiences today (e.g., in experimental theater or location-based entertainment contexts). Bad News is a realized game enjoyed by players with varying degrees of performance experience, and won the Innovative Game Design track of the 2016 ACM Conference on Human Factors in Computing Systems (CHI) Student Game Competition.

International Conference on Interactive Digital Storytelling, Nov 2016

Interactive storytelling applications have at their disposal massive numbers of human-authored st... more Interactive storytelling applications have at their disposal massive numbers of human-authored stories, in the form of narrative weblog posts, from which story content could be harvested and repurposed. Such repurposing is currently inhibited, however, in that many blog narratives are not sufficiently coherent for use in these applications. In a narrative that is not coherent, the order of the events in the narrative is not clear given the text of the story. We present the results of a study exploring automatic methods for estimating the coherence of narrative blog posts. In the end, our simplest model—one that only considers the degree to which story text is capitalized and punctuated—vastly outperformed a baseline model and, curiously, a series of more sophisticated models. Future work may use this simple model as a baseline, or may use it along with the classifier that it extends to automatically extract large numbers of narrative blog posts from the web for purposes such as interactive storytelling.

International Conference on Interactive Digital Storytelling, Nov 2016

We present Expressionist, an authoring tool for in-game text generation that combines the raw gen... more We present Expressionist, an authoring tool for in-game text generation that combines the raw generative power of context-free grammars (CFGs) with the expressive power of free-text markup. Specifically, authors use the tool to define CFGs whose nonterminal symbols may be annotated using arbitrary author-defined tagsets. Any content generated by the CFG comes packaged with explicit metadata in the form of the markup attributed to all the symbols that were expanded to produce the content. Expressionist has already been utilized in two released games and it is currently being used in two ongoing projects. In this paper, we describe the tool and discuss these usage examples in a series of case studies. Expressionist is planned for release in late 2016.

Experimental AI in Games, Oct 2016

Computationally assisted performance is a burgeoning area for AI applications, and an important s... more Computationally assisted performance is a burgeoning area for AI applications, and an important stepping stone toward the dream of generative and personalized narrative experiences. As more pieces of computationally assisted performance are developed, it will become ever more important to develop a vocabulary with which to describe them. Inspired by previous work in creating taxonomies for other related domains, this paper outlines a taxonomy for performance-based experiences, drawn from digital games, traditional theatre, and the hybrid of the two. Having such a taxonomy not only creates a common language with which to discuss such experiences, but reveals unexplored design space in the field, and the particular applications of artificial intelligence necessary to realize them.

Experimental AI in Games, Oct 2016

We present Juke Joint, a small work of interactive storytelling that demonstrates an extension to... more We present Juke Joint, a small work of interactive storytelling that demonstrates an extension to the Talk of the Town framework by which characters form thoughts, expressed in natural language, that are elicited by environmental stimuli. Juke Joint takes place in a procedurally generated American small town, in a bar with a haunted jukebox and two patrons facing personal dilemmas; the player is a ghost whose only action is to select which song from the jukebox will play. As the lyrics of the song emanate from the machine, thoughts are elicited in the minds of the patrons, constituting streams of consciousness that may eventually lead them to resolutions of their respective dilemmas. In this paper, we outline the game and also the AI architecture that makes it possible; the latter combines a light simulation of stimulus processing with a novel approach to natural language generation.

We present perhaps the first exploration of the procedural generation of gameworld languages, mea... more We present perhaps the first exploration of the procedural generation of gameworld languages, meaning fictional languages spoken by characters in a game's diegesis. This preliminary work takes a simulation-based approach in which languages are represented abstractly , using a vectorial scheme, and evolve over simulated game time as the emergent byproduct of diegetic agent interactions. While this method does not produce concrete languages with surface representations and rules, the abstract vectors that it does produce still provide interesting authorial affordances, which we discuss. Moreover, as an operationalization of linguistic theories, particularly Labov's incrementation model, we position our work as a potential contribution to the computational modeling of linguistic phenomena.

5th Workshop on Social Believability in Games

We present a method for generating social networks for gameworlds with very many characters. The ... more We present a method for generating social networks for gameworlds with very many characters. The method, a generalization of the approach we employ in Talk of the Town, operates from a simple principle: characters' affinities for one another evolve as a function of the compatibility of their personalities and the amount of time they spend together. By this principle, friendships emerge as compatible characters interact more extensively and enmities emerge as incompatible characters do so. Beyond platonic affinity, our method evolves romantic feelings by the same principle (applied to romantic considerations). How exactly compatibility is defined (e.g., by operationalizing psychological, sociological, or artistic theories) may vary according to the goals of a particular project. In this paper, we describe our method in generalized terms that are agnostic to our specific application of it, so that interested readers may implement it in their own systems.

5th Workshop on Social Believability in Games

We have developed a videogame dialogue manager that, when fed onto itself, offers a nice authoria... more We have developed a videogame dialogue manager that, when fed onto itself, offers a nice authorial affordance: the ability to trigger generative, procedural conversations among non-player characters (NPCs). By feeding the system onto itself, we mean that NPCs converse with one another by virtue of the same dialogue manager—that is, a single system selects dialogue for both conversants using the same policies for turn-taking and content selection. This makes such conversations fully procedural (as opposed to branching dialogue) and allows them to play out automatically, with no need for player input. While our dialogue manager is described at length elsewhere, in this short paper we will discuss how we plan to use it for background believability and storytelling in the game that houses it, Talk of the Town.

CHI Conference on Human Factors in Computing Systems, 2020

2nd Workshop on Natural Language Processing for Conversational AI, 2020

International Conference on the Foundations of Digital Games, 2020

Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove diffi... more Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial study comparing it with traditional web-based educational materials from an existing university RCR course. The results highlight that utilizing a choice-based interactive story game is more effective for RCR education, with learners developing significantly higher engagement, stronger overall moral reasoning skills, and better knowledge scores for certain RCR topics.

PhD Thesis, University of California, Santa Cruz, 2018

10th International Conference on Interactive Digital Storytelling, 2017

AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Oct 10, 2016

Intelligent Virtual Agents, Sep 2016

University of California, Santa Cruz, Technical Report UCSC-SOE-16-11, Jul 22, 2016

International Conference on Interactive Digital Storytelling, Nov 2016

Experimental AI in Games, Oct 2016

5th Workshop on Social Believability in Games

CHI Conference on Human Factors in Computing Systems, 2020