Learning affordances for categorizing objects and their properties (original) (raw)

A system for learning basic object affordances using a self-organizing map

2008

When a cognitive system encounters particular objects, it needs to know what effect each of its possible actions will have on the state of each of those objects in order to be able to make effective decisions and achieve its goals. Moreover, it should be able to generalize effectively so that when it encounters novel objects, it is able to estimate what effect its actions will have on them based on its experiences with previously encountered similar objects. This idea is encapsulated by the term "affordance", e.g. "a ball affords being rolled to the right when pushed from the left." In this paper, we discuss the development of a cognitive vision platform that uses a robotic arm to interact with household objects in an attempt to learn some of their basic affordance properties. We outline the various sensor and effector module competencies that were needed to achieve this and describe an experiment that uses a self-organizing map to integrate these modalities in a working affordance learning system.

Unsupervised learning of basic object affordances from object properties

2009

Affordance learning has, in recent years, been generating heightened interest in both the cognitive vision and developmental robotics communities. In this paper we describe the development of a system that uses a robotic arm to interact with household objects on a table surface while observing the interactions using camera systems. Various computer vision methods are used to derive, firstly, object property features from intensity images and range data gathered before interaction and, subsequently, result features derived from video sequences gathered during and after interaction. We propose a novel affordance learning algorithm that automatically discretizes the result feature space in an unsupervised manner to form affordance classes that are then used as labels to train a supervised classifier in the object property feature space. This classifier may then be used to predict affordance classes, grounded in the result space, of novel objects based on object property observations.

Learning intermediate object affordances: Towards the development of a tool concept

4th International Conference on Development and Learning and on Epigenetic Robotics, 2014

Inspired by the extraordinary ability of young infants to learn how to grasp and manipulate objects, many works in robotics have proposed developmental approaches to allow robots to learn the effects of their own motor actions on objects, i.e., the objects affordances. While holding an object, infants also promote its contact with other objects, resulting in object-object interactions that may afford effects not possible otherwise. Depending on the characteristics of both the held object (intermediate) and the acted object (primary), systematic outcomes may occur, leading to the emergence of a primitive concept of tool. In this paper we describe experiments with a humanoid robot exploring object-object interactions in a playground scenario and learning a probabilistic causal model of the effects of actions as functions of the characteristics of both objects. The model directly links the objects' 2D shape visual cues to the effects of actions. Because no object recognition skills are required, generalization to novel objects is possible by exploiting the correlations between the shape descriptors. We show experiments where an affordance model is learned in a simulated environment, and is then used on the real robotic platform, showing generalization abilities in effect prediction. We argue that, despite the fact that during exploration no concept of tool is given to the system, this very concept may emerge from the knowledge that intermediate objects lead to significant effects when acting on other objects.

Towards learning basic object affordances from object properties

2008

The capacity for learning to recognize and exploit environmental affordances is an important consideration for the design of current and future developmental robotic systems. We present a system that uses a robotic arm, camera systems and self-organizing maps to learn basic affordances of objects.

Affordances Provide a Fundamental Categorization Principle for Visual Scenes

How do we know that a kitchen is a kitchen by looking? Relatively little is known about how we conceptualize and categorize different visual environments. Traditional models of visual perception posit that scene categorization is achieved through the recognition of a scene's objects, yet these models cannot account for mounting evidence that human observers are relatively insensitive to the local details in an image. Psychologists have long theorized that the affordances, or the actionable possibilities of a stimulus are pivotal to its perception. To what extent are scene categories created from similar affordances? Using a large-scale experiment using hundreds of scene categories, we show that the activities afforded by a visual scene provide a fundamental categorization principle. Affordance-based similarity explained the majority of the structure in human scene categorization patterns, outperforming alternative similarities based on objects or visual features. When all these models are combined, affordances provide the majority of the predictive power in the combined model, and nearly half of the total explained variance is captured only by affordances. These results challenge many existing models of high-level visual perception, and provide immediately testable hypotheses for the functional organization of the human perceptual system. Significance Statement How do we know that a kitchen is a kitchen by looking? Models of visual perception assume that scene identification is facilitated through object recognition. However, these models fail to account for observers' relative insensitivity to local image details. We explore an alternative view that posits that a scene's identity is determined by the possibilities for actions that a scene affords (its affordances). In a large-scale experiment using hundreds of scene categories, we found that human scene similarity ratings were more closely related to affordance-based similarity than to object or visual feature-based models. Combining models revealed that nearly half of the explained variance was captured only by affordances. This work demonstrates that affordances provide a fundamental grouping principle for scenes. V.

Learning Intermediate Features of Object Affordances with a Convolutional Neural Network

2020

Our ability to interact with the world around us relies on being able to infer what actions objects afford -- often referred to as affordances. The neural mechanisms of object-action associations are realized in the visuomotor pathway where information about both visual properties and actions is integrated into common representations. However, explicating these mechanisms is particularly challenging in the case of affordances because there is hardly any one-to-one mapping between visual features and inferred actions. To better understand the nature of affordances, we trained a deep convolutional neural network (CNN) to recognize affordances from images and to learn the underlying features or the dimensionality of affordances. Such features form an underlying compositional structure for the general representation of affordances which can then be tested against human neural data. We view this representational analysis as the first step towards a more formal account of how humans perce...

Using a SOFM to learn Object Affordances

2004

Learning affordances can be defined as learning action potentials, i.e., learning that an object exhibiting certain regularities offers the possibility of performing a particular action. We propose a method to endow an agent with the capability of acquiring this knowledge by relating the object invariants with the potentiality of performing an action via interaction episodes with each object. We introduce a biologically inspired model to test this learning hypothesis and a set of experiments to check its validity in a Webots simulator with a Khepera robot in a simple environment. The experiment set aims to show the use of a GWR network to cluster the sensory input of the agent; furthermore, that the aforementioned algorithm for neural clustering can be used as a starting point to build agents that learn the relevant functional bindings between the cues in the environment and the internal needs of an agent.

Software Model of Autonomous Object Affordances Learning

2008

Abstract Learning to recognize affordances is an essential skill essential for safe autonomous operation and intelligent planning. In this thesis, we present a general learning algorithm for affordances that combines an active learning approach with decision tree induction–smart exploration with rule extraction. Our framework constructs a mental model of objects' affordances both through knowledge discovery and knowledge transfer scenarios in both propositional and relational domains.

Functional Object Class Detection Based on Learned Affordance Cues

Lecture Notes in Computer Science, 2008

Current approaches to visual object class detection mainly focus on the recognition of basic level categories, such as cars, motorbikes, mugs and bottles. Although these approaches have demonstrated impressive performance in terms of recognition, their restriction to these categories seems inadequate in the context of embodied, cognitive agents. Here, distinguishing objects according to functional aspects based on object affordances is important in order to enable manipulation of and interaction between physical objects and cognitive agent.