A Comparison Between a Two-Feedback Control Loop and a Reinforcement Learning Algorithm for Compliant Low-Cost Series Elastic Actuators (original) (raw)
Related papers
2019 5th International Conference on Control, Automation and Robotics (ICCAR)
Even though a few examples of elastic snake robots exist, they are generally expensive and tailored to custom-made hardware/software components that are not openly available. In this work, Serpens, a newly-designed low-cost, open-source and highly-compliant modular snake robot with series elastic actuator (SEA) is presented. Serpens features precision torque control and stereoscopic vision. Only low-cost commercial-off-the-shelf (COTS) components are adopted. The robot modules can be 3Dprinted by using Fused Deposition Modelling (FDM) manufacturing technology, thus making the rapid-prototyping process very economical and fast. A screw-less assembly mechanism allows for connecting the modules and reconfigure the robot in reliable and robust manner. By using a low-cost sensing approach, functions for torque sensing at the joint level, sensitive collision detection and joint compliant control are possible. The concept of modularity is also applied to the system architecture on both the software and hardware sides. The software architecture is based on the Robot Operating System (ROS). This paper describes the design of Serpens and presents preliminary simulation and experimental results which illustrate its potential.
Reinforcement Learning of a CPG-regulated Locomotion Controller for a Soft Snake Robot
arXiv (Cornell University), 2022
Intelligent control of soft robots is challenging due to the nonlinear and difficult-to-model dynamics. One promising model-free approach for soft robot control is reinforcement learning (RL). However, model-free RL methods tend to be computationally expensive and data-inefficient and may not yield natural and smooth locomotion patterns for soft robots. In this work, we develop a bio-inspired design of a learning-based goaltracking controller for a soft snake robot. The controller is composed of two modules: An RL module for learning goaltracking behaviors given the unmodeled and stochastic dynamics of the robot, and a central pattern generator (CPG) with the Matsuoka oscillators for generating stable and diverse locomotion patterns. We theoretically investigate the maneuverability of Matsuoka CPG's oscillation bias, frequency, and amplitude for steering control, velocity control, and sim-to-real adaptation of the soft snake robot. Based on this analysis, we proposed a composition of RL and CPG modules such that the RL module regulates the tonic inputs to the CPG system given state feedback from the robot, and the output of the CPG module is then transformed into pressure inputs to pneumatic actuators of the soft snake robot. This design allows the RL agent to naturally learn to entrain the desired locomotion patterns determined by the CPG maneuverability. We validated the optimality and robustness of the control design in both simulation and real experiments, and performed extensive comparisons with state-ofart RL methods to demonstrate the benefit of our bio-inspired control design.
Real-Time Adaptive Control of a Flexible Manipulator Using Reinforcement Learning
This paper exploits reinforcement learning (RL) for developing real-time adaptive control of tip trajectory and deflection of a two-link flexible manipulator handling variable payloads. This proposed adaptive controller consists of a proportional derivative (PD) tracking loop and an actor-critic-based RL loop that adapts the actor and critic weights in response to payload variations while suppressing the tip deflection and tracking the desired trajectory. The actor-critic-based RL loop uses a recursive least square (RLS)-based temporal difference (TD) learning with eligibility trace and an adaptive memory to estimate the critic weights and a gradient-based estimator for estimating actor weights. Tip trajectory tracking and suppression of tip deflection performances of the proposed RL-based adaptive controller (RLAC) are compared with that of a nonlinear regression-based direct adaptive controller (DAC) and a fuzzy learning-based adaptive controller (FLAC). Simulation and experimental results envisage that the RLAC outperforms both the DAC and FLAC.
Motion Planning and Iterative Learning Control of a Modular Soft Robotic Snake
Frontiers in Robotics and AI, 2020
Snake robotics is an important research topic with a wide range of applications, including inspection in confined spaces, search-and-rescue, and disaster response. Snake robots are well-suited to these applications because of their versatility and adaptability to unstructured and constrained environments. In this paper, we introduce a soft pneumatic robotic snake that can imitate the capabilities of biological snakes, its soft body can provide flexibility and adaptability to the environment. This paper combines soft mobile robot modeling, proprioceptive feedback control, and motion planning to pave the way for functional soft robotic snake autonomy. We propose a pressure-operated soft robotic snake with a high degree of modularity that makes use of customized embedded flexible curvature sensing. On this platform, we introduce the use of iterative learning control using feedback from the on-board curvature sensors to enable the snake to automatically correct its gait for superior loc...
The Redesigned Serpens, a Low-Cost, Highly Compliant Snake Robot
Robotics, 2022
The term perception-driven obstacle-aided locomotion (POAL) was proposed to describe locomotion in which a snake robot leverages a sensory-perceptual system to exploit the surrounding operational environment and to identify walls, obstacles, or other structures as a means of propulsion. To attain POAL from a control standpoint, the accurate identification of push-points and reliable determination of feasible contact reaction forces are required. This is difficult to achieve with rigidly actuated robots because of the lack of compliance. As a possible solution to this challenge, our research group recently presented Serpens, a low-cost, open-source, and highly compliant multi-purpose modular snake robot with a series elastic actuator (SEA). In this paper, we propose a new prototyping iteration for our snake robot to achieve a more dependable design. The following three contributions are outlined in this work as a whole: the remodelling of the elastic joint with the addition of a damp...
Reinforcement Learning Based Adaptive Control of a Flexible Manipulator
In this paper a new nonlinear adaptive controller using actor-critic based Reinforcement Learning (RL) is proposed to adapt the load pickup and release operation while following a desired trajectory by the end effector for a Two-Link Flexible Manipulator (TLFM). Simulation results show that the proposed RL based adaptive control gives better trajectory tracking performance and suppression of link vibration compared to conventional adaptive controllers with time varying payload.
Repetitive learning with fuzzy logic adaptive control of a flexible robot manipulator
IFAC Proceedings Volumes, 2003
Operational problems with robot manipulators in space relate to several factors, one most importantly being structural flexibility and subsequently significant difficulties with the control systems, especially, position control. A control strategy is devised for positioning the endpoint of a two-link robot manipulator modeled with assumed modes flexible dynamics repetitively tracking a square trajectory. The dominant assumed modes of vibration are determined for Euler-Bernoulli cantilever beam boundary conditions then, coupled with the nonlinear dynamics for rigid links to form an Euler-Lagrange inverse flexible dynamics robot model. A Jacobian transpose control law actuates the robot links. While repetitive tracking alone achieves no improvement in control precision, adapting the control law by a fuzzy logic system achieves consistent tracking precision.
Model Reference Learning Control for Rigid Robots
The equations of motion of a rigid robot are often known only approximately, as some of the parameters are not known exactly and there are also unmodelled nonlinearities. Most adaptive control schemes can estimate the parameters if the structure of the equations is known, but are not very useful if structure itself is not known. In this paper we propose a model reference learning control scheme using Adaptive Network based Fuzzy Inference System (ANFIS) for control of rigid robots whose model may h a ve p a r ametric and structural uncertainties. The approximate model of a robot, which m a y di er very signi cantly from the actual robot in parametric values and structure, is used as a reference plant and a nonlinear model based controller is designed based on this model. The ANFIS corrector provides an additional correction to control input as a function of the present and desired states of the plant. The error between states of plant and that of reference plant is used to tune the ANFIS corrector. The proposed control scheme has been implemented for a two-degree-of-freedom serial rigid robot. The results of the simulation experiments carried out show that the proposed control scheme can learn to control the unmodelled dynamics. The AN-FIS controller is shown to give improved performance for parameter as well as structural uncertainties.
Snake robot locomotion in a cluttered environment where the snake robot utilises a sensory-perceptual system to perceive the surrounding operational environment for means of propulsion is defined as perception-driven obstacle-aided locomotion (POAL). From a control point of view, achieving POAL with traditional rigidly-actuated robots is challenging because of the complex interaction between the snake robot and the immediate environment. To simplify the control complexity, compliant motion and fine torque control on each joint is essential. Accordingly, intrinsically elastic joints have become progressively prominent over the last years for a variety robotic applications. Commonly, elastic joints are considered to outperform rigid actuation in terms of peak dynamics, robustness, and energy efficiency. Even though a few examples of elastic snake robots exist, they are generally expensive to manufacture and tailored to custom-made hardware/software components that are not openly available off-the-shelf. In this work, Serpens, a newly-designed low-cost, open-source and highly-compliant multipurpose modular snake robot with series elastic actuator (SEA) is presented. Serpens features precision torque control and stereoscopic vision. Only low-cost commercial-off-the-shelf (COTS) components are adopted. The robot modules can be 3D-printed by using Fused Deposition Modelling (FDM) manufacturing technology, thus making the rapid-prototyping process very economical and fast. A screw-less assembly mechanism allows for connecting the modules and reconfigure the robot in a very reliable and robust manner. The concept of modularity is also applied to the system architecture on both the software and hardware sides. Each module is independent, being controlled by a self-reliant controller board. The software architecture is based on the Robot Operating System (ROS). This paper describes the design of Serpens and presents preliminary simulation and experimental results, which illustrate its performance.
Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot
arXiv (Cornell University), 2020
In this paper, we present a new locomotion control method for soft robot snakes. Inspired by biological snakes, our control architecture is composed of two key modules: A deep reinforcement learning (RL) module for achieving adaptive goal-tracking behaviors with changing goals, and a central pattern generator (CPG) system with Matsuoka oscillators for generating stable and diverse locomotion patterns. The two modules are interconnected into a closed-loop system: The RL module, analogizing the locomotion region located in the midbrain of vertebrate animals, regulates the input to the CPG system given state feedback from the robot. The output of the CPG system is then translated into pressure inputs to pneumatic actuators of the soft snake robot. Based on the fact that the oscillation frequency and wave amplitude of the Matsuoka oscillator can be independently controlled under different time scales, we further adapt the option-critic framework to improve the learning performance measured by optimality and data efficiency. The performance of the proposed controller is experimentally validated with both simulated and real soft snake robots.