Learning modular and transferable forward models of the motions of push manipulated objects (original) (raw)

Learning Transferable Push Manipulation Skills in Novel Contexts

Frontiers in Neurorobotics, 2021

This paper is concerned with learning transferable forward models for push manipulation that can be applying to novel contexts and how to improve the quality of prediction when critical information is available. We propose to learn a parametric internal model for push interactions that, similar for humans, enables a robot to predict the outcome of a physical interaction even in novel contexts. Given a desired push action, humans are capable to identify where to place their finger on a new object so to produce a predictable motion of the object. We achieve the same behaviour by factorising the learning into two parts. First, we learn a set of local contact models to represent the geometrical relations between the robot pusher, the object, and the environment. Then we learn a set of parametric local motion models to predict how these contacts change throughout a push. The set of contact and motion models represent our internal model. By adjusting the shapes of the distributions over t...

Learning to predict the behaviour of deformable objects through and for robotic interaction

2013

Every day environments contain a great variety of deformable objects and it is not possible to program a robot in advance to know about their characteristic behaviours. For this reason, robots have been highly successful in manoeuvring deformable objects mainly in the industrial sector, where the types of interactions are predictable and highly restricted, but research in everyday environments remains largely unexplored. The contributions of this thesis are: i) the application of an elastic/plastic mass-spring method to model and predict the behaviour of deformable objects manipulated by a robot; ii) the automatic calibration of the parameters of the model, using images of real objects as ground truth; iii) the use of piece-wise regression curves to predict the reaction forces, and iv) the use of the output of this force prediction model as input for the mass-spring model which in turn predicts object deformations; v) the use of the obtained models to solve a material classification...

A vision-based learning method for pushing manipulation

1993

Abstract--We describe an unsupervised on-line method for learning of manipulative actions that allows a robot to push an object connected to it with a rotational point contact to a desired point in image-space. By observing tile results of its actions on tile object's orientation in imagespace, the system forms a predictive forward empirical model. This acquired model is used on-line for manipulation planning and control as it improves. Rather than explicitly inverting the forward model to achieve trajectory control, a stochastic action ...

Learning Visual Dynamics Models of Rigid Objects using Relational Inductive Biases10-30

2019

Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work, we learn action-conditional forward dynamics models of a simulated manipulation task from visual observations involving cluttered and irregularly shaped objects. We investigate two GNN approaches and empirically assess their capability to generalize to scenarios with novel and an increasing number of objects. The first, Graph Networks (GN) based approach, considers explicitly defined edge attributes and not only does it consistently underperform an auto-encoder baseline that we modified to predict future states, our results indicate how different edge attributes can significantly influence the predictions. Consequently, we develop the AutoPredictor that does not rely ...