On Curiosity in Intelligent Robotic Systems (original) (raw)

The playground experiment: Task-independent development of a curious robot

Proceedings of the AAAI …, 2005

This paper presents the mechanism of Intelligent Adaptive Curiosity. This is an intrinsic motivation system which pushes the robot towards situations in which it maximizes its learning progress. It makes the robot focus on situations which are neither too predictable nor too unpredictable. This mechanism is a source of autonomous mental development for the robot: the complexity of its activities autonomously increases and a developmental sequence appears without being manually constructed. We test this motivation system on a real robot which evolves on a baby playmat with objects that it can learn to manipulate. We show that it first spends time in situations which are easy to learn, then shifts progressively its attention to situations of increasing difficulty, avoiding situations in which nothing can be learnt.

Intrinsic motivation systems for autonomous mental development

… , IEEE Transactions on, 2007

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot's activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology.

Intrinsically motivated developmental learning of communication in robotic agents

2012

This thesis is concerned with the emergence of communication in artificial agents as an integrated part of a more general developmental progression. We demonstrate how early gestural communication can emerge out of sensorimotor exploration before moving on to linguistic communication. We then show how communicative abilities can feed back into more general motor learning. We take a cumulative developmental approach, with two different robotic platforms undergoing a series of psychologically inspired developmental stages. These begin with the robot learning about its own body’s capabilities and limitations, then on to object interaction, the learning of proto-imperative pointing and early language learning. Finally this culminates in more complex object interaction in the form of learning to build stacks of objects with the linguistic capabilities developed earlier being used to help guide the robot’s learning. This developmental progression is supported by a schema learning mechanism which constructs a hierarchy of competencies capable of dealing with problems of gradually increasing complexity. To allow for the learning of general concepts we introduce an algorithm for the generalisation of schemas from a small number of examples through parameterisation. Throughout the robot’s development its actions are driven by an intrinsic motivation system designed to mimic the play-like behaviour seen in infants. We suggest a possible approach to intrinsic motivation in a schema learning system and demonstrate how this can lead to the rapid unsupervised learning of both specific experiences and general concepts.

Intelligent Adaptive Curiosity: a source of Self-Development

2004

This paper presents the mechanism of Intelligent Adaptive Curiosity. This is a drive which pushes the robot towards situations in which it maximizes its learning progress. It makes the robot focus on situations which are neither too predictable nor too unpredictable. This mechanism is a source of selfdevelopment for the robot: the complexity of its activity autonomously increases. Indeed, we show that it first spends time in situations which are easy to learn, then shifts progressively its attention to situations of increasing difficulty, avoiding situations in which nothing can be learnt.

The Benefits of Boredom: an Exploration in Developmental Robotics

Artificial Life, 2007

Self-directed learning is an essential component of artificial and biological intelligent systems that are required to interact with and adapt to complex real world environments. Inspired by psychological and neuroscientific data, many algorithms and architectures have been proposed in the field of developmental robotics that use novelty as a training signal. Such approaches are aimed at motivating the exploration of sensory-motor contingencies for which mental models have not yet been accurately formed, driving the agent to develop taskindependent competencies (such as understanding object affordances) without the need for explicit teaching. However, novelty-driven exploration on its own leads to a number of wellknown problems that impede competence acquisition such as the attraction of agents to chaotic or unlearnable tasks and the temporary oversampling of aspects of the environment until they are no longer novel. This paper contributes to the field, taking insight from neuroscientific data on selective attention (particularly the temporary "boredom" associated with recently seen stimuli and a counter preference for the familiar), to propose mechanisms that may help address the noted problems relating to developmental learning in robots. Experiments conducted on an AIBO ERS-7 robotic dog demonstrate the potential of the approach.

Intrinsic motivation systems for open-ended development

An emergent framework for self-motivation in developmental robotics

2004

Abstract This paper explores a philosophy and connectionist algorithm for creating a long-term, self-motivated developmental robot control system. Self-motivation is viewed as an emergent property arising from two competing pressures: the need to accurately predict the environment while simultaneously wanting to seek out novelty in the environment. These competing internal pressures are designed to drive the system in a manner reminiscent of a co-evolutionary arms race.

A curiosity model for artificial agents

Proceedings of the 14th European Conference on Artificial Life ECAL 2017, 2017

Curiosity is an inherent characteristic of the animal instinct, which stimulates the need to obtain further knowledge and leads to the exploration of the surrounding environment. In this document we present a computational curiosity model, which aims at simulating that kind of behavior on artificial agents. This model is influenced by the two main curiosity theories defended by psychologists-Curiosity Drive Theory and Optimal Arousal Model. By merging both theories, as well as aspects from other sources, we concluded that curiosity can be defined in terms of the agent's personality, its level of arousal, and the interest of the object of curiosity. The interest factor is defined in terms of the importance of the object of curiosity to the agent's goals, its novelty, and surprise. To assess the performance of the model in practice, we designed a scenario consisting of virtual agents exploring a tile-based world, where objects may exist. The performance of the model in this scenario was evaluated in incremental steps, each one introducing a new component to the model. Furthermore, in addition to empirical evaluation, the model was also subjected to evaluation by human observers. The results obtained from both sources show that our model is able to simulate curiosity on virtual agents and that each of the identified factors has its role in the simulation.

Exploration and Curiosity in Robot Learning and Inference

2011

This report documents the program and the outcomes of Dagstuhl Seminar 11131 "Exploration and Curiosity in Robot Learning and Inference". This seminar was concerned with answering the question: how should a robot choose its actions and experiences so as to maximise the effectiveness of its learning?. The seminar brought together workers from three fields: machine learning, robotics and computational neuroscience. The seminar gave an overview of active research, and identified open research problems. In particular the seminar identified the difficulties in moving from theoretically well grounded notions of curiosity to practical robot implementations. Seminar 27. March -1. April, 2011 -www.dagstuhl.de/11131 1998 ACM Subject Classification I.2.9 Robotics

On Curiosity in Intelligent Robotic Systems (original) (raw)

Related papers