Marilyn Walker - Academia.edu (original) (raw)

Papers by Marilyn Walker

Conversation is an essential component of social behaviour, one of the primary means by which hum... more Conversation is an essential component of social behaviour, one of the primary means by which humans express emotions, moods, attitudes and personality. Thus a key technical capability for dialogue applications, such as interactive narrative systems (INS), human robot interaction (HRI) and spoken dialogue systems (SDS), is the ability to support natural conversational interaction. However, system utterances in existing systems are typically handcrafted, leading to problems of portability and scalability. We propose a framework for automatically generating language projecting different personality traits based on the 'Big Five' model of personality. We show that our PERSONAGE generator can produce utterances with recognisable personality for all Big Five traits, according to human judges. We also test the ability of PERSONAGE to vary the characters' personality in an existing interactive narrative system, showing that some forms of variation can be automatically obtained in a new domain, depending on the level of utterance representation.

Conversation is an essential component of social behavior, one of the primary means by which huma... more Conversation is an essential component of social behavior, one of the primary means by which humans express intentions, beliefs, emotions, attitudes and personality. Thus the development of systems to support natural conversational interaction has been a long term research goal. In natural conversation, humans adapt to one another across many levels of utterance production via processes variously described as linguistic style matching, entrainment, alignment, audience design, and accommodation. A number of recent studies strongly suggest that dialogue systems that adapted to the user in a similar way would be more effective. However, a major research challenge in this area is the ability to dynamically generate user-adaptive utterance variations. As part of a personality-based user adaptation framework, this article describes Personage, a highly parameterizable generator which provides a large number of parameters to support adaptation to a user's linguistic style. We show how we can systematically apply results from psycholinguistic studies that document the linguistic reflexes of personality, in order to develop models to control Personage's parameters, and produce utterances matching particular personality profiles. When we evaluate these outputs with human judges, the results indicate that humans perceive the personality of system utterances in the way that the system intended.

Spoken dialogue systems promise efficient and natural access to information services from any pho... more Spoken dialogue systems promise efficient and natural access to information services from any phone. Recently, spoken dialogue systems for widely used applications such as email, travel information, and customer care have moved from research labs into commercial use. These applications can receive millions of calls a month. This huge amount of spoken dialogue data has led to a need for fully automatic methods for selecting a subset of caller dialogues that are most likely to be useful for further system improvement, to be stored, transcribed and further analyzed. This paper reports results on automatically training a Problematic Dialogue Identifier to classify problematic human-computer dialogues using a corpus of 1242 DARPA Communicator dialogues in the travel planning domain. We show that using fully automatic features we can identify classes of problematic dialogues with accuracies from 67% to 89%.

Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), 2016

This paper discusses the motivation behind common knowledge. Common knowledge has been argued to ... more This paper discusses the motivation behind common knowledge. Common knowledge has been argued to be necessary for joint action in general and for language use as a particular kind of joint action. However, this term has been broadly interpreted. Two major issues must be addressed: (1) What mental state corresponds to common knowledge, i.e., is knowledge, belief or supposition the appropriate mental attitude? (2) What inference process allows agents to achieve common knowledge?

Interactive Narrative often involves dialogue with virtual dramatic characters. In this paper we ... more Interactive Narrative often involves dialogue with virtual dramatic characters. In this paper we compare two kinds of models of character style: one based on models derived from the Big Five theory personality, and the other derived from a corpus-based method applied to characters and films from the IMSDb archive. We apply these models to character utterances for a pilot narrative-based outdoor augmented reality game called Murder in the Arboretum. We use an objective quantitative metric to estimate the quality of a character model, with the aim of predicting model quality without perceptual experiments. We show that corpus-based character models derived from individual characters are often more detailed and specific than personality based models, but that there is a strong correlation between personality judgments of original character dialogue and personality judgments of utterances generated for Murder in the Arboretum that use the derived character models.

Proceedings of the Workshop on Languages in Social Media, Jun 23, 2011

ABSTRACT The recent proliferation of political and social forums has given rise to a wealth of fr... more ABSTRACT The recent proliferation of political and social forums has given rise to a wealth of freely accessible naturalistic arguments. People can "talk" to anyone they want, at any time, in any location, about any topic. Here we use a Mechanical Turk annotated corpus of forum discussions as a gold standard for the recognition of disagreement in online ideological forums. We analyze the utility of meta-post features, contextual features, dependency features and word-based features for signaling the disagreement relation. We show that using contextual and dialogic features we can achieve accuracies up to 68% as compared to a unigram baseline of 63%.

Meaningful choice has often been identified as a key component in a player's engagement with an i... more Meaningful choice has often been identified as a key component in a player's engagement with an interactive narrative, but branching stories require tremendous amounts of hand-authored content, in amounts that increase exponentially rather than linearly as more choice points are added. Previous approaches to reducing authorial burden for computer RPGs have relied on creating better tools to manage existing unwieldy structures of quests and dialogue trees. We hypothesize that reducing authorial burden and increasing agency are two sides of the same coin, requiring specific advancements in two related areas of design and technology research: (1) dynamic story management architecture that represents story events abstractly and allows story elements to be selected and re-ordered in response to player choices, and (2) dynamic dialogue generation to allow a single story event to be revealed differently by different characters and in the context of dynamic relationships between those characters and the player. This paper describes SpyFeet, a playable prototype of a storytelling system designed to test this hypothesis.

Proceedings of the 2012 Conference of the North American Chapter of the Association For Computational Linguistics Human Language Technologies, Jun 3, 2012

... Marilyn A. Walker, Pranav Anand, Robert Abbott and Ricky Grant Baskin School of Engineering &... more ... Marilyn A. Walker, Pranav Anand, Robert Abbott and Ricky Grant Baskin School of Engineering & Linguistics Department University of California Santa Cruz Santa Cruz, Ca. ... Association for Computational Linguistics. JW Pennebaker, LE Francis, and RJ Booth, 2001. ...

Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media, 2014

ABSTRACT In _Using Language_, Herbert Clark proposes a broadly integrative theory of language as ... more ABSTRACT In _Using Language_, Herbert Clark proposes a broadly integrative theory of language as action. The book examines both social and cognitive aspects of language use, drawing from speech act theory (Austion 1965; Searle 1965; Allen and Perrault 1980), theories of discourse and dialogue (Reichman 1985; Grosz and Sidner 1986), and theories of social interaction (Goffman 1970; Brown and Levinson 1987; Sacks, Schegloff, and Jefferson 1974). In order to integrate these different perspectives, the book relies on empirical research by Clark and his students over the last fifteen years.

This document is a proposal of research intended to complete a Ph.D. in Computer Science. The ove... more This document is a proposal of research intended to complete a Ph.D. in Computer Science. The overall goal of the proposed work is to demonstrate a connection between agents as limited reasoners and the use of informationally redundant utterances in problem-solving dialogues. This document describes some long range objectives and some preliminary results toward this goal. Comments from readers on the proposed work would be most welcome.

This paper addresses three topics on functionality that should be incorporated into planning form... more This paper addresses three topics on functionality that should be incorporated into planning formalisms so that they can: (1) respect limitations of the recipient's working memory; (2) rely on the recipient's inferences to achieve communicative goals; and (3) deliberately violate normally-respected conversational rules to achieve a communicative eect. The argument is that a planner for communicative acts must rely on a model of the recipient's working memory. This is supported by data from a corpus of naturally-occurring problem-solving dialogues, and by results from computational experiments. The computational experiments show that the performance of resource limited agents is determined by their choice of communicative acts.

... ZDV Denver ZAB Albuquerque ZFW Fort Worth ZAU Chicago ZOB Cleveland ZTL Atlanta ... Page 12. ... more ... ZDV Denver ZAB Albuquerque ZFW Fort Worth ZAU Chicago ZOB Cleveland ZTL Atlanta ... Page 12. 2-2 URET Utilization 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% May-99 Jun-99 Jul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 Feb-00 Mar-00 Apr-00 % Used ...

Seventh Artificial Intelligence and Interactive Digital Entertainment Conference, 2011

Many forms of interactive digital entertainment involve interacting with virtual dramatic charact... more Many forms of interactive digital entertainment involve interacting with virtual dramatic characters. Our long term goal is to procedurally generate character dialogue behavior that automatically mimics, or blends, the style of existing characters. In this paper, we show how linguistic elements in character dialogue can define the style of characters in our RPG SpyFeet. We utilize a corpus of 862 film scripts from the IMSDb website, representing 7,400 characters, 664,000 lines of dialogue and 9,599,000 word tokens. We utilize counts of linguistic reflexes that have been used previously for personality or author recognition to discriminate different character types. With classification experiments, we show that different types of characters can be distinguished at accuracies up to 83% over a baseline of 20%. We discuss the characteristics of the learned models and show how they can be used to mimic particular film characters.

Conversation is an essential component of social behaviour, one of the primary means by which hum... more Conversation is an essential component of social behaviour, one of the primary means by which humans express emotions, moods, attitudes and personality. Thus a key technical capability for dialogue applications, such as interactive narrative systems (INS), human robot interaction (HRI) and spoken dialogue systems (SDS), is the ability to support natural conversational interaction. However, system utterances in existing systems are typically handcrafted, leading to problems of portability and scalability. We propose a framework for automatically generating language projecting different personality traits based on the 'Big Five' model of personality. We show that our PERSONAGE generator can produce utterances with recognisable personality for all Big Five traits, according to human judges. We also test the ability of PERSONAGE to vary the characters' personality in an existing interactive narrative system, showing that some forms of variation can be automatically obtained in a new domain, depending on the level of utterance representation.

Conversation is an essential component of social behavior, one of the primary means by which huma... more Conversation is an essential component of social behavior, one of the primary means by which humans express intentions, beliefs, emotions, attitudes and personality. Thus the development of systems to support natural conversational interaction has been a long term research goal. In natural conversation, humans adapt to one another across many levels of utterance production via processes variously described as linguistic style matching, entrainment, alignment, audience design, and accommodation. A number of recent studies strongly suggest that dialogue systems that adapted to the user in a similar way would be more effective. However, a major research challenge in this area is the ability to dynamically generate user-adaptive utterance variations. As part of a personality-based user adaptation framework, this article describes Personage, a highly parameterizable generator which provides a large number of parameters to support adaptation to a user's linguistic style. We show how we can systematically apply results from psycholinguistic studies that document the linguistic reflexes of personality, in order to develop models to control Personage's parameters, and produce utterances matching particular personality profiles. When we evaluate these outputs with human judges, the results indicate that humans perceive the personality of system utterances in the way that the system intended.

Spoken dialogue systems promise efficient and natural access to information services from any pho... more Spoken dialogue systems promise efficient and natural access to information services from any phone. Recently, spoken dialogue systems for widely used applications such as email, travel information, and customer care have moved from research labs into commercial use. These applications can receive millions of calls a month. This huge amount of spoken dialogue data has led to a need for fully automatic methods for selecting a subset of caller dialogues that are most likely to be useful for further system improvement, to be stored, transcribed and further analyzed. This paper reports results on automatically training a Problematic Dialogue Identifier to classify problematic human-computer dialogues using a corpus of 1242 DARPA Communicator dialogues in the travel planning domain. We show that using fully automatic features we can identify classes of problematic dialogues with accuracies from 67% to 89%.

Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), 2016

This paper discusses the motivation behind common knowledge. Common knowledge has been argued to ... more This paper discusses the motivation behind common knowledge. Common knowledge has been argued to be necessary for joint action in general and for language use as a particular kind of joint action. However, this term has been broadly interpreted. Two major issues must be addressed: (1) What mental state corresponds to common knowledge, i.e., is knowledge, belief or supposition the appropriate mental attitude? (2) What inference process allows agents to achieve common knowledge?

Interactive Narrative often involves dialogue with virtual dramatic characters. In this paper we ... more Interactive Narrative often involves dialogue with virtual dramatic characters. In this paper we compare two kinds of models of character style: one based on models derived from the Big Five theory personality, and the other derived from a corpus-based method applied to characters and films from the IMSDb archive. We apply these models to character utterances for a pilot narrative-based outdoor augmented reality game called Murder in the Arboretum. We use an objective quantitative metric to estimate the quality of a character model, with the aim of predicting model quality without perceptual experiments. We show that corpus-based character models derived from individual characters are often more detailed and specific than personality based models, but that there is a strong correlation between personality judgments of original character dialogue and personality judgments of utterances generated for Murder in the Arboretum that use the derived character models.

Proceedings of the Workshop on Languages in Social Media, Jun 23, 2011

ABSTRACT The recent proliferation of political and social forums has given rise to a wealth of fr... more ABSTRACT The recent proliferation of political and social forums has given rise to a wealth of freely accessible naturalistic arguments. People can "talk" to anyone they want, at any time, in any location, about any topic. Here we use a Mechanical Turk annotated corpus of forum discussions as a gold standard for the recognition of disagreement in online ideological forums. We analyze the utility of meta-post features, contextual features, dependency features and word-based features for signaling the disagreement relation. We show that using contextual and dialogic features we can achieve accuracies up to 68% as compared to a unigram baseline of 63%.

Meaningful choice has often been identified as a key component in a player's engagement with an i... more Meaningful choice has often been identified as a key component in a player's engagement with an interactive narrative, but branching stories require tremendous amounts of hand-authored content, in amounts that increase exponentially rather than linearly as more choice points are added. Previous approaches to reducing authorial burden for computer RPGs have relied on creating better tools to manage existing unwieldy structures of quests and dialogue trees. We hypothesize that reducing authorial burden and increasing agency are two sides of the same coin, requiring specific advancements in two related areas of design and technology research: (1) dynamic story management architecture that represents story events abstractly and allows story elements to be selected and re-ordered in response to player choices, and (2) dynamic dialogue generation to allow a single story event to be revealed differently by different characters and in the context of dynamic relationships between those characters and the player. This paper describes SpyFeet, a playable prototype of a storytelling system designed to test this hypothesis.

Proceedings of the 2012 Conference of the North American Chapter of the Association For Computational Linguistics Human Language Technologies, Jun 3, 2012

... Marilyn A. Walker, Pranav Anand, Robert Abbott and Ricky Grant Baskin School of Engineering &... more ... Marilyn A. Walker, Pranav Anand, Robert Abbott and Ricky Grant Baskin School of Engineering & Linguistics Department University of California Santa Cruz Santa Cruz, Ca. ... Association for Computational Linguistics. JW Pennebaker, LE Francis, and RJ Booth, 2001. ...

Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media, 2014

ABSTRACT In _Using Language_, Herbert Clark proposes a broadly integrative theory of language as ... more ABSTRACT In _Using Language_, Herbert Clark proposes a broadly integrative theory of language as action. The book examines both social and cognitive aspects of language use, drawing from speech act theory (Austion 1965; Searle 1965; Allen and Perrault 1980), theories of discourse and dialogue (Reichman 1985; Grosz and Sidner 1986), and theories of social interaction (Goffman 1970; Brown and Levinson 1987; Sacks, Schegloff, and Jefferson 1974). In order to integrate these different perspectives, the book relies on empirical research by Clark and his students over the last fifteen years.

This document is a proposal of research intended to complete a Ph.D. in Computer Science. The ove... more This document is a proposal of research intended to complete a Ph.D. in Computer Science. The overall goal of the proposed work is to demonstrate a connection between agents as limited reasoners and the use of informationally redundant utterances in problem-solving dialogues. This document describes some long range objectives and some preliminary results toward this goal. Comments from readers on the proposed work would be most welcome.

This paper addresses three topics on functionality that should be incorporated into planning form... more This paper addresses three topics on functionality that should be incorporated into planning formalisms so that they can: (1) respect limitations of the recipient's working memory; (2) rely on the recipient's inferences to achieve communicative goals; and (3) deliberately violate normally-respected conversational rules to achieve a communicative eect. The argument is that a planner for communicative acts must rely on a model of the recipient's working memory. This is supported by data from a corpus of naturally-occurring problem-solving dialogues, and by results from computational experiments. The computational experiments show that the performance of resource limited agents is determined by their choice of communicative acts.

... ZDV Denver ZAB Albuquerque ZFW Fort Worth ZAU Chicago ZOB Cleveland ZTL Atlanta ... Page 12. ... more ... ZDV Denver ZAB Albuquerque ZFW Fort Worth ZAU Chicago ZOB Cleveland ZTL Atlanta ... Page 12. 2-2 URET Utilization 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% May-99 Jun-99 Jul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 Feb-00 Mar-00 Apr-00 % Used ...

Seventh Artificial Intelligence and Interactive Digital Entertainment Conference, 2011

Many forms of interactive digital entertainment involve interacting with virtual dramatic charact... more Many forms of interactive digital entertainment involve interacting with virtual dramatic characters. Our long term goal is to procedurally generate character dialogue behavior that automatically mimics, or blends, the style of existing characters. In this paper, we show how linguistic elements in character dialogue can define the style of characters in our RPG SpyFeet. We utilize a corpus of 862 film scripts from the IMSDb website, representing 7,400 characters, 664,000 lines of dialogue and 9,599,000 word tokens. We utilize counts of linguistic reflexes that have been used previously for personality or author recognition to discriminate different character types. With classification experiments, we show that different types of characters can be distinguished at accuracies up to 83% over a baseline of 20%. We discuss the characteristics of the learned models and show how they can be used to mimic particular film characters.