Towards an indexical model of situated language comprehension for real-world cognitive agents (original) (raw)
Related papers
ArXiv, 2016
We propose a computational model of situated language comprehension based on the Indexical Hypothesis that generates meaning representations by translating amodal linguistic symbols to modal representations of beliefs, knowledge, and experience external to the linguistic system. This Indexical Model incorporates multiple information sources, including perceptions, domain knowledge, and short-term and long-term experiences during comprehension. We show that exploiting diverse information sources can alleviate ambiguities that arise from contextual use of underspecific referring expressions and unexpressed argument alternations of verbs. The model is being used to support linguistic interactions in Rosie, an agent implemented in Soar that learns from instruction.
Context for language understanding by intelligent agents
Applied Ontology, 2019
This paper describes the layers of context leveraged by language-endowed intelligent agents (LEIAs) during incremental natural language understanding (NLU). Context is defined as a combination of (a) the perceptual stimuli available to the agent at the given point in time, and (b) the knowledge elements and reasoning activated at the given stage of the agent's interpretation of those stimuli. This approach to NLU addresses the treatment of a large number of difficult linguistic phenomena that are essential for high-quality NLU but are not being tackled by the knowledge-lean approaches that are typical of modernday natural language processing. Although LEIAs are being developed as components of prototype application systems, this paper is not about implementations or evaluations-its contribution is conceptual, with everything described applicable to any artificial intelligent agent environment.
Linguistically Competent Intelligent Agents: Towards a Digital Agora
Information retrieval
Abstract—The potential of a globally interconnected and immediately and widely accessible knowledge network is immense. In our view, the challenges in building this digital agora lie in the design of intelligent agents that can reason and communicate in natural language. There are a number of technical challenges in this endeavor, as this effort involves the integration of a number of key technologies, such as natural language understanding (NLU), concept-based information retrieval and the formalization of commonsense reasoning. In ...
Situated Comprehension of Imperative Sentences in Embodied, Cognitive Agents
2012
Linguistic communication relies on non-linguistic context to convey meaning. That context might include, for instance, recent or long-term experience, semantic knowledge of the world, or objects and events in the immediate environment. In this paper, we describe embodied agents instantiated in Soar cognitive architecture that use context derived from their linguistic, perceptual, procedural and semantic knowledge for comprehending imperative sentences.
1997
After having compared four methods of studying language, I will examine their respective consequences with regard to architecture natural language understanding and generation. I will concentrate in this paper mainly on two points: how and when to use pragmatic knowledge for automatic text understanding. I will then show how to overcome the inadequacies of the control mechanisms traditionally used in Natural Language Understanding systems, and the usefulness of multi-expert systems for maintaining modularity without introducing artificial ambiguities. In addition, I propose some extensions to the classical reflectivity in order to make natural language understanding systems reflective. Taking all these factors into account led to CARAMEL-1: a general natural language understanding system, which was meant to be general enough in order to apply to various kinds of applications (The acronym CARAMEL stands for "Compréhension Automatique de Récits, Apprentissage et Modélisation des Échanges Langagiers" meaning roughly "Automatic Story Understanding, Learning and Dialogue Management"). One of the problems that arise in a large modular architecture is to control explicitly all the processes. However, this is not only cumbersome, but also it does not account for automatic reflex processes, yet these latter are essential for natural language understanding. This is why I have introduced a new kind of memory model (the Sketchboard) which implies not only various kinds of relations between the processes but allows also for reactive feedback loops.
Grounding Language for Interactive Task Learning
Proceedings of the First Workshop on Language Grounding for Robotics, 2017
This paper describes how language is grounded by a comprehension system called Lucia within a robotic agent called Rosie that can manipulate objects and navigate indoors. The whole system is built within the Soar cognitive architecture and uses Embodied Construction Grammar (ECG) as a formalism for describing linguistic knowledge. Grounding is performed using knowledge from the grammar itself, from the linguistic context, from the agent's perception, and from an ontology of long-term knowledge about object categories and properties and actions the agent can perform. The paper also describes a benchmark corpus of 200 sentences in this domain, along with test versions of the world model and ontology, and gold-standard meanings for each of the sentences. The benchmark is contained in the supplemental materials.
Towards Situated, Interactive, Instructable Agents in a Cognitive Architecture
2011
This paper discusses the challenge of designing instructable agents that can learn through interaction with a human expert. Learning through instruction is a powerful paradigm for acquiring knowledge because it limits the complexity of the learning task in a variety of ways. To support learning through instruction, the agent must be able to effectively communicate its lack of knowledge to the human, comprehend instructions, and apply them to the ongoing task. We identify some problems of concern when designing instructable agents. We propose an agent design that addresses some of these problems. We instantiate this design in the Soar cognitive architecture and analyze its capabilities on a learning task.
Cognitively-inspired representational approach to meaning in machine dialogue
Knowledge-Based Systems, 2014
One of the most fundamental research questions in the field of human-machine interaction is how to 29 enable dialogue systems to capture the meaning of spontaneously produced linguistic inputs without 30 explicit syntactic expectations. This paper introduces a cognitively-inspired representational model 31 intended to address this research question. To the extent that this model is cognitively-inspired, it 32 integrates insights from behavioral and neuroimaging studies on working memory operations and lan-33 guage-impaired patients (i.e., Broca's aphasics). The level of detail contained in the specification of the 34 model is sufficient for a computational implementation, while the level of abstraction is sufficient to 35 enable generalization of the model over different interaction domains. Finally, the paper reports on a 36 domain-independent framework for end-user programming of adaptive dialogue management modules. 37 Ó 2014 Published by Elsevier B.V. 38 39 the users should not be forced to intentionally adapt their dialogue acts to a preset grammar. Instead, they should be allowed, as far as possible, to express themselves naturally. It is in line with a recent Wizard-of-Oz study reported by Gnjatović and Rösner [24, p. 141] showing that nonlinguistic context (e.g., a graphical display, etc.) shared between the subjects and the simulated system influences 57 the language of the subjects to a high extent with respect to fre-58 quency of ''ungrammatical'' utterances, e.g., elliptical, minor, con-59 text-dependent utterances, etc. Hence, a dialogue system should 60 be able to cope with such dialogue phenomena. 61 The paper is divided in two main parts. The first part (Sections 62 2-5) introduces a cognitively-inspired model for meaning represen-63 tation. We draw upon and integrate insights from behavioral and 64 neuroimaging studies on working memory operations and lan-65 guage-impaired patients (i.e., Broca's aphasics) in order to specify 66 the storage and processing aspects of the model. In this respect, 67 our approach contributes to the field of cognitive infocommunica-68 tions [2,34]. The level of detail contained in the specification of 69 the model is sufficient for a computational implementation, while 70 the level of abstraction is sufficient to enable generalization of the 71 model over different interaction domains. The second part of the 72 paper (Sections 6 and 7) discusses the computational appropriate-73 ness and generalizability of the model, and reports on a domain-74 independent framework for end-user programming of adaptive 75 dialogue management modules. 76 2. Background and related work 77 This paper integrates and expands upon previous work on the 78 research question of modeling attentional information in task-79 oriented human-machine interaction. The focus tree is a model of 80 attentional state introduced by Gnjatović and colleagues
The Realis Model of Human Interpreters and Its Application in Computational Linguistics
2010
As we strive for sophisticated machine translation and reliable information extraction, we have launched a subproject pertaining to modelling human interpreters. The model is based on ReALIS, a new “postMontagovian” discourse-semantic theory concerning the formal interpretation of sentences constituting coherent discourses, with a lifelong model of lexical, interpersonal and cultural / encyclopedic knowledge of interpreters in its center including their reciprocal knowledge on each other. After the introduction of ReALIS, we provide linguistic data in order to show that intelligent language processing requires a realistic model of human interpreters. Then we put down some principles of the implementation (in progress) and demonstrate how to apply our model in computational linguistics. 1 REALIS: THE THEORY IN THE BACKGROUND ReALIS, REciprocal And Lifelong Interpretation System, is a new “post-Montagovian” (Kamp et al. 2005) theory concerning the formal interpretation of sentences co...