Invited Talks (original) (raw)

Not cheating on the Turing Test: towards grounded language learning in Artificial Intelligence

Master's Thesis, 2020

In this thesis, I carry out a novel and interdisciplinary analysis of various complex factors involved in human natural-language acquisition, use and comprehension, aimed at uncovering some of the basic requirements for if we were to try and develop artificially intelligent (AI) agents with similar capacities. Inspired by a recent publication wherein I explored the complexities and challenges involved in enabling AI systems to deal with the grammatical(i.e. syntactic and morphological) irregularities and ambiguities inherent in natural language (Alberts, 2019), I turn my focus here towards appropriately inferring the content of symbols themselves—as ‘grounded’ in real-world percepts, actions, and situations. I first introduce the key theoretical problems I aim to address in theories of mind and language. For background, I discuss the co-development of AI and the controverted strands of computational theories of mind in cognitive science, and the grounding problem(or ‘internalist trap’) faced by them. I then describe the approach I take to address the grounding problem in the rest of the thesis. This proceeds in chapter I. To unpack and address the issue, I offer a critical analysis of the relevant theoretical literature in philosophy of mind, psychology, cognitive science and (cognitive) linguistics in chapter II. I first evaluate the major philosophical/psychological debates regarding the nature of concepts; theories regarding how concepts are acquired, used, and represented in the mind; and, on that basis, offer my own account of conceptual structure, grounded in current (cognitively plausible) connectionist theories of thought. To further explicate how such concepts are acquired and communicated, I evaluate the relevant embodied (e.g. cognitive, perceptive, sensorimotor, affective, etc.) factors involved in grounded human (social) cognition, drawing from current scientific research in the areas of4E Cognition and social cognition. On that basis, I turn my focus specifically towards grounded theories of language, drawing from the cognitive linguistics programme that aims to develop a naturalised, cognitively plausible understanding of human concept/language acquisition and use. I conclude the chapter with a summary wherein I integrate my findings from these various disciplines, presenting a general theoretical basis upon which to evaluate more practical considerations for its implementation in AI—the topic of the following chapter. In chapter III, I offer an overview of the different major approaches(and their integrations) in the area of Natural Language Understanding in AI, evaluating their respective strengths and shortcomings in terms of specific models. I then offer a critical summary wherein I contrast and contextualise the different approaches in terms of the more fundamental theoretical convictions they seem to reflect. On that basis, in the final chapter, I re-evaluate the aforementioned grounding problem and the different ways in which it has been interpreted in different (theoretical and practical) disciplines, distinguishing between a stronger and weaker reading. I then present arguments for why implementing the stronger version in AI seems, both practically and theoretically, problematic. Instead, drawing from the theoretical insights I gathered, I consider some of the key requirements for ‘grounding’ (in the weaker sense) as much as possible of natural language use with robotic AI agents, including implementational constraints that might need to be put in place to achieve this. Finally, I evaluate some of the key challenges that may be involved, if indeed the aim were to meet all the requirements specified.

A First-Order-Logic Based Model for Grounded Language Learning

Advances in Intelligent Data Analysis XIV, 2015

Much is still unknown about how children learn language, but it is clear that they perform "grounded" language learning: they learn the grammar and vocabulary not just from examples of sentences, but from examples of sentences in a particular context. Grounded language learning has been the subject of much research. Most of this work focuses on particular aspects, such as constructing semantic parsers, or on particular types of applications. In this paper, we take a broader view that includes an aspect that has received little attention until now: learning the meaning of phrases from phrase/context pairs in which the phrase's meaning is not explicitly represented. We propose a simple model for this task that uses first-order logic representations for contexts and meanings, including a simple incremental learning algorithm. We experimentally demonstrate that the proposed model can explain the gradual learning of simple concepts and language structure, and that it can easily be used for interpretation, generation, and translation of phrases.

Bootstrapping grounded word semantics

Linguistic evolution through language …, 2002

The paper reports on experiments with a population of visually grounded robotic agents capable of bootstrapping their own ontology and shared lexicon without prior design nor other forms of human intervention. The agents do so while playing a particular language game called the guessing game. We show that synonymy and ambiguity arise as emergent properties in the lexicon, due to the situated grounded character of the agent-environment interaction, but that there are also tendencies to dampen them so as to make the language more coherent and thus more optimal from the viewpoints of communicative success, cognitive complexity, and learnability.

Learning grounded semantics with word trees: Prepositions and pronouns

2007 IEEE 6th International Conference on Development and Learning, 2007

The authors present a method by which a robot can learn the meanings of words from unlabeled correct examples in context. The "word trees" method consists of reconstructing the speaker's decision process in choosing a word. The facts about an object and its relation to other objects that maximally reduce the uncertainty (entropy) of word choice become the decision nodes of this tree. The conjunction of the choices leading to a word becomes its logical definition. Definitions thereby become only as complex as is necessary to distinguish words in the vocabulary, making the method appear to follow a heuristic that developmental psychologists call the "Principle of Contrast." Combined with a method for inferring word type and reference, the method produces semantics complete enough to produce or understand full sentences. The method was implemented on a robot with visual, auditory, and positional sensors, and succeeded in learning the differences between "I," "you," "he," "this," "that," "above," "below," and "near."

Grounded Learning of Grammatical Constructions

AAAI Spring Symp. On Learning Grounded …, 2001

We describe a model of grammar learning in which all linguistic units are grounded in rich conceptual representations, and larger grammatical constructions involve relational mappings between form and meaning that are built up from smaller (e.g., lexical) constructions. The algorithm we describe for acquiring these grammatical constructions consists of three separate but interacting processes: an analysis procedure that uses the current set of constructions to identify mappings between an utterance and its accompanying situation; a hypothesis procedure that creates new constructions to account for remaining correlations between these two domains; and reorganization processes that generalize existing constructions on the basis of similarity and cooccurrence. The algorithm is thus grounded not only in the twin poles of form and meaning but also, more importantly, in the relational mappings between the two.

Open-ended Grounded Semantics

2010

Artificial agents trying to achieve communicative goals in situated interactions in the real-world need powerful computational systems for conceptualizing their environment. In order to provide embodied artificial systems with rich semantics reminiscent of human language complexity, agents need ways of both conceptualizing complex compositional semantic structure and actively reconstructing semantic structure, due to uncertainty and ambiguity in transmission. Furthermore, the systems must be open-ended and adaptive and allow agents to adjust their semantic inventories in order to reach their goals. This paper presents recent progress in modeling open-ended, grounded semantics through a unified software system that addresses these problems.

Acquiring Grounded Representations of Words with Situated Interactive Instruction

2012

We present an approach for acquiring grounded representations of words from mixed-initiative, situated interactions with a human instructor. The work focuses on the acquisition of diverse types of knowledge including perceptual, semantic, and procedural knowledge along with learning grounded meanings. Interactive learning allows the agent to control its learning by requesting instructions about unknown concepts, making learning efficient. Our approach has been instantiated in Soar and has been evaluated on a table-top robotic arm capable of manipulating small objects. 1. The robot and the perception/actuation system was developed by Edwin Olson at University of Michigan.

Unsupervised Selection of Negative Examples for Grounded Language Learning

2018

There has been substantial work in recent years on grounded language acquisition, in which a model is learned that relates linguistic constructs to the perceivable world. While powerful, this approach is frequently hindered by ambiguities and omissions found in natural language. One such omission is the lack of negative descriptions of objects. We describe an unsupervised system that learns visual classifiers associated with words, using semantic similarity to automatically choose negative examples from a corpus of perceptual and linguistic data. We evaluate the effectiveness of each stage as well as the system’s performance on the overall learning task.

Building Language-Agnostic Grounded Language Learning Systems

2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2019

Learning the meaning of grounded languagelanguage that references a robot's physical environment and perceptual data-is an important and increasingly widely studied problem in robotics and human-robot interaction. However, with a few exceptions, research in robotics has focused on learning groundings for a single natural language pertaining to rich perceptual data. We present experiments on taking an existing natural language grounding system designed for English and applying it to a novel multilingual corpus of descriptions of objects paired with RGB-D perceptual data. We demonstrate that this specific approach transfers well to different languages, but also present possible design constraints to consider for grounded language learning systems intended for robots that will function in a variety of linguistic settings.

Situated grounded word semantics

INTERNATIONAL JOINT CONFERENCE ON …, 1999

The paper reports on experiments in which autonomous visually grounded agents bootstrap an ontology and a shared lexicon without prior design nor other forms of human intervention. The agents do so while playing a particular language game called the guessing game. We show that synonymy and polysemy arise as emergent properties in the language but also that there are tendencies to dampen it so as to make the language more coherent and thus more optimal from the viewpoints of communicative success, cognitive complexity, and learnability.