Robotic Arm Control and Task Training through Deep Reinforcement Learning (original) (raw)
Related papers
A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation
Sensors
Robotic manipulation challenges, such as grasping and object manipulation, have been tackled successfully with the help of deep reinforcement learning systems. We give an overview of the recent advances in deep reinforcement learning algorithms for robotic manipulation tasks in this review. We begin by outlining the fundamental ideas of reinforcement learning and the parts of a reinforcement learning system. The many deep reinforcement learning algorithms, such as value-based methods, policy-based methods, and actor–critic approaches, that have been suggested for robotic manipulation tasks are then covered. We also examine the numerous issues that have arisen when applying these algorithms to robotics tasks, as well as the various solutions that have been put forth to deal with these issues. Finally, we highlight several unsolved research issues and talk about possible future directions for the subject.
Sim–Real Mapping of an Image-Based Robot Arm Controller Using Deep Reinforcement Learning
Applied Sciences
Models trained with Deep Reinforcement learning (DRL) have been deployed in various areas of robotics with varying degree of success. To overcome the limitations of data gathering in the real world, DRL training utilizes simulated environments and transfers the learned policy to real-world scenarios, i.e., sim–real transfer. Simulators fail to accurately capture the entire dynamics of the real world, so simulation-trained policies often fail when applied to reality, termed a reality gap (RG). In this paper, we propose a search (mapping) algorithm that takes in real-world observation (images) and maps them to the policy-equivalent images in the simulated environment using a convolution neural network (CNN) model. The two-step training process, DRL policy and a mapping model, overcomes the RG problem with simulated data only. We evaluated the proposed system with a gripping task of a custom-made robot arm in the real world and compared the performance against a conventional DRL sim–re...
Australasian Conference on Robotics and Automation, 2023
Applications using Model-Free Reinforcement Learning (MFRL) have grown exponentially and have shown remarkable results in the last decade. The application of MFRL to robots shows significant promise for its capability to solve complex control problems, at least virtually or in simulation. Due to the practical challenges of training in a real-world environment, there is limited work bridging the gap to real physical robots. This article benchmarks the state-of-the-art MFRL algorithms training on an open-source robotic manipulation testbed consisting of a fully actuated, 4-Degrees of Freedom (DoF), two-fingered robot gripper to understand the limitations and challenges involved in real-world applications. Experimental analysis using two different statespace representations is presented to understand their impact on executing a dexterous manipulation task. The source code, the CAD files of the robotic manipulation testbed, and a handy guide on how to approach MFRL's application to real-world are provided to facilitate replication of the results and further experimentation by other research groups.
Deep Reinforcement Learning for Large Scale Robotic Simulations
2019
Deep reinforcement learning enables algorithms to learn complex behavior, deal with continuous action spaces and find good strategies in environments with high dimensional state spaces. With deep reinforcement learning being an active area of research and many concurrent inventions, we decided to focus on a simple robotic task to evaluate a set of ideas that might help to solve recent reinforcement learning problems. The focus is on enabling distributed set up to execute and run experiments with the least amount of time and benefit from the available computational power. Another focus is on the transferability between different physics engines, where we experiment on how to use a trained agent from one environment into another different environment with a different physics engine. The purpose of this thesis is to unify the differences between reinforcement learning environments by implementing simple abstract classes between the selected environments which can be extended to support more environment. With this, the focus was on setting and enabling distribution for training to reduce the time of the experiment. We select two of the state of the art reinforcement learning methods to train, evaluate and test the distributed and transferability. The goal of this strategy is to reduce training time and eventually help the algorithms to scale, collect experiences, and train the agents effectively. The concluding evaluation and results prove the general applicability of the described concepts by testing them using selected environments. In our experiments, the effect of distribution was shown in the training time between the experiments. Furthermore, the last performed experiment we demonstrate how to use transfer learning and trained agents in a new learning environment. These concepts might be reused for future experiments.
Navigation of Robotic-Arms using Policy Gradient Reinforcement Learning
International Journal of Computing and Digital Systems, 2022
In this paper, the Deep Deterministic Policy Gradient (DDPG) reinforcement-learning algorithm is employed to enable a double-jointed robot arm to reach continuously changing target locations. The experimentation of the algorithm is carried out by training an agent to control the movement of this double-jointed robot arm. The architectures of the actor and critic networks are meticulously designed and the DDPG hyperparameters are carefully tuned. An enhanced version of the DDPG is also presented to handle multiple robot arms simultaneously. The trained agents are successfully tested in the Unity Machine Learning Agents environment for controlling both a single robot arm as well as multiple simultaneous robot arms. The testing shows the robust performance of the DDPG algorithm for empowering robot arm maneuvering in complex environments.
International Journal of Intelligent Robotics and Applications, 2020
For robot manipulation, reinforcement learning has provided an effective end to end approach in controlling the complicated dynamic system. Model-free reinforcement learning methods ignore the model of system dynamics and are limited to simple behavior control. By contrast, model-based methods can quickly reach optimal trajectory planning by building a dynamic system model. However, it is not easy to build an accurate and efficient system model with high generalization ability, especially when facing complex dynamic system and various manipulation tasks. Furthermore, when the rewards provided by the environment are sparse, the agent will also lose effective guidance and fail to optimize the policy efficiently, which results in considerably decreased sample efficiency. In this paper, a model-based deep reinforcement learning algorithm, in which a deep neural network model is utilized to simulate the system dynamics, is designed for robot manipulation. The proposed deep neural network model is robust enough to deal with complex control tasks and possesses the generalization ability. Moreover, a curiosity-based experience replay method is incorporated to solve the sparse reward problem and improve the sample efficiency in reinforcement learning. The agent who manipulates a robotic hand, will be encouraged to explore optimal trajectories according to the failure experience. Simulation experiment results show great effectiveness of the proposed method. Various manipulation tasks are achieved successfully in such a complex dynamic system and the sample efficiency gets improved even in a sparse reward environment, as the learning time gets reduced considerably.
Deep Reinforcement Learning with Interactive Feedback in a Human–Robot Environment
Applied Sciences, 2020
Robots are extending their presence in domestic environments every day, it being more common to see them carrying out tasks in home scenarios. In the future, robots are expected to increasingly perform more complex tasks and, therefore, be able to acquire experience from different sources as quickly as possible. A plausible approach to address this issue is interactive feedback, where a trainer advises a learner on which actions should be taken from specific states to speed up the learning process. Moreover, deep reinforcement learning has been recently widely used in robotics to learn the environment and acquire new skills autonomously. However, an open issue when using deep reinforcement learning is the excessive time needed to learn a task from raw input images. In this work, we propose a deep reinforcement learning approach with interactive feedback to learn a domestic task in a Human–Robot scenario. We compare three different learning methods using a simulated robotic arm for t...
Applied Sciences
While working side-by-side, humans and robots complete each other nowadays, and we may say that they work hand in hand. This study aims to evolve the grasping task by reaching the intended object based on deep reinforcement learning. Thereby, in this paper, we propose a deep deterministic policy gradient approach that can be applied to a numerous-degrees-of-freedom robotic arm towards autonomous objects grasping according to their classification and a given task. In this study, this approach is realized by a five-degrees-of-freedom robotic arm that reaches the targeted object using the inverse kinematics method. You Only Look Once v5 is employed for object detection, and backward projection is used to detect the three-dimensional position of the target. After computing the angles of the joints at the detected position by inverse kinematics, the robot’s arm is moved towards the target object’s emplacement thanks to the algorithm. Our approach provides a neural inverse kinematics solu...
DRL: Deep Reinforcement Learning for Intelligent Robot Control - Concept, Literature, and Future
ArXiv, 2021
Combination of machine learning (for generating machine intelligence), computer vision (for better environment perception), and robotic systems (for controlled environment interaction) motivates this work toward proposing a vision-based learning framework for intelligent robot control as the ultimate goal (vision-based learning robot). This work specifically introduces deep reinforcement learning as the the learning framework, a General-purpose framework for AI (AGI) meaning application-independent and platform-independent. In terms of robot control, this framework is proposing specifically a high-level control architecture independent of the low-level control, meaning these two required level of control can be developed separately from each other. In this aspect, the high-level control creates the required intelligence for the control of the platform using the recorded low-level controlling data from that same platform generated by a trainer. The recorded low-level controlling data...
Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control
Manipulation in highly dynamic and complex environments is challenging for robots. This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from camera images and without any prior knowledge is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation of the ma-nipulator. In simulation, a Deep Q Network (DQN) was demonstrated to perform target reaching after training. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images generated by a simulator according to real-time robot joint angles.