Position Control of a Mobile Robot through Deep Reinforcement Learning (original) (raw)
Related papers
Deep Reinforcement Learning Based Mobile Robot Navigation in Unknown Indoor Environments
International Conference on Interdisciplinary Applications of Artificial Intelligence (ICIDAAI), 2021
The importance of autonomous robots has been increasing day by day with the development of technology. Difficulties in performing many tasks such as target recognition, navigation, and obstacle avoidance autonomously by mobile robots are problems that must be overcome. In recent years, the use of deep reinforcement learning algorithms in robot navigation has been increasing. One of the most important reasons why deep reinforcement learning is preferred over traditional algorithms is that robots can learn the environments by themselves without any prior knowledge or map in environments with obstacles. This study proposes a navigation system based on the dueling deep Q network algorithm, which is one of the deep reinforcement learning algorithms, for a mobile robot in an unknown environment to reach its target by avoiding obstacles. In the study, a 2D laser sensor and an RGB-D camera has been used so that the mobile robot can detect and recognize the static and dynamic obstacles in front of itself, and its surroundings. Robot Operating System (ROS) and Gazebo simulator have been used to model the robot and environment. The experiment results show that the mobile robot can reach its targets by avoiding static and dynamic obstacles in unknown environments.
Robot Navigation through the Deep Q-Learning Algorithm
The paper aims to present the results of an assessment of adherence to the Deep Q-learning algorithm, applied to a vehicular navigation robot. The robot's job was to transport parts through an environment, for this purpose, a decision system was built based on the Deep Q-learning algorithm, with the aid of an artificial neural network that received data from the sensors as input and allowed autonomous navigation in an environment. For the experiments, the mobile robot-maintained communication via the network with other robotic components present in the environment through the MQTT protocol.
Mobile Robot Application with Hierarchical Start Position DQN
Computational Intelligence and Neuroscience
Advances in deep learning significantly affect reinforcement learning, which results in the emergence of Deep RL (DRL). DRL does not need a data set and has the potential beyond the performance of human experts, resulting in significant developments in the field of artificial intelligence. However, because a DRL agent has to interact with the environment a lot while it is trained, it is difficult to be trained directly in the real environment due to the long training time, high cost, and possible material damage. Therefore, most or all of the training of DRL agents for real-world applications is conducted in virtual environments. This study focused on the difficulty in a mobile robot to reach its target by making a path plan in a real-world environment. The Minimalistic Gridworld virtual environment has been used for training the DRL agent, and to our knowledge, we have implemented the first real-world implementation for this environment. A DRL algorithm with higher performance than...
Mobile robots interacting with obstacles control based on artificial intelligence
Proceedings of the Sixth International Conference on Research in Intelligent and Computing, 2022
In this paper, research on the applications of artificial intelligence in implementing Deep Deterministic Policy Gradient (DDPG) on Gazebo model and the reality of mobile robot has been studied and applied. The goal of the experimental studies is to navigate the mobile robot to learn the best possible action to move in real-world environments when facing fixed and mobile obstacles. When the robot moves in an environment with obstacles, the robot will automatically control to avoid these obstacles. Then, the more time that can be maintained within a specific limit, the more rewards are accumulated and therefore better results will be achieved. The authors performed various tests with many transform parameters and proved that the DDPG algorithm is more efficient than algorithms like Q-learning, Machine learning, deep Q-network, etc. Then execute SLAM to recognize the robot positions, and virtual maps are precisely built and displayed in Rviz. The research results will be the basis for the design and construction of control algorithms for mobile robots and industrial robots applied in programming techniques and industrial factory automation control.
DRL: Deep Reinforcement Learning for Intelligent Robot Control - Concept, Literature, and Future
ArXiv, 2021
Combination of machine learning (for generating machine intelligence), computer vision (for better environment perception), and robotic systems (for controlled environment interaction) motivates this work toward proposing a vision-based learning framework for intelligent robot control as the ultimate goal (vision-based learning robot). This work specifically introduces deep reinforcement learning as the the learning framework, a General-purpose framework for AI (AGI) meaning application-independent and platform-independent. In terms of robot control, this framework is proposing specifically a high-level control architecture independent of the low-level control, meaning these two required level of control can be developed separately from each other. In this aspect, the high-level control creates the required intelligence for the control of the platform using the recorded low-level controlling data from that same platform generated by a trainer. The recorded low-level controlling data...
DEEP REINFORCEMENT LEARNING FOR NAVIGATION IN CLUTTERED ENVIRONMENTS
Collision-free motion is essential for mobile robots. Most approaches to collision-free and efficient navigation with wheeled robots require parameter tuning by experts to obtain good navigation behavior. In this paper, we aim at learning an optimal navigation policy by deep reinforcement learning to overcome this manual parameter tuning. Our approach uses proximal policy optimization to train the policy and achieve collision-free and goal-directed behavior. The output of the learned network are the robot's trans-lational and angular velocities for the next time step. Our method combines path planning on a 2D grid with reinforcement learning and does not need any supervision. Our network is first trained in a simple environment and then transferred to scenarios of increasing complexity. We implemented our approach in C++ and Python for the Robot Operating System (ROS) and thoroughly tested it in several simulated as well as real-world experiments. The experiments illustrate that our trained policy can be applied to solve complex navigation tasks. Furthermore, we compare the performance of our learned controller to the popular dynamic window approach (DWA) of ROS. As the experimental results show, a robot controlled by our learned policy reaches the goal significantly faster compared to using the DWA by closely bypassing obstacles and thus saving time.
arXiv (Cornell University), 2023
In this work, we present two Deep Reinforcement Learning (Deep-RL) approaches to enhance the problem of mapless navigation for a terrestrial mobile robot. Our methodology focus on comparing a Deep-RL technique based on the Deep Q-Network (DQN) algorithm with a second one based on the Double Deep Q-Network (DDQN) algorithm. We use 24 laser measurement samples and the relative position and angle of the agent to the target as information for our agents, which provide the actions as velocities for our robot. By using a low-dimensional sensing structure of learning, we show that it is possible to train an agent to perform navigation-related tasks and obstacle avoidance without using complex sensing information. The proposed methodology was successfully used in three distinct simulated environments. Overall, it was shown that Double Deep structures further enhance the problem for the navigation of mobile robots when compared to the ones with simple Q structures.
Dynamic Obstacle Avoidance Technique for Mobile Robot Navigation Using Deep Reinforcement Learning
International Journal of Emerging Trends in Engineering Research , 2023
In the realm of mobile robotics, navigating around obstacles is a fundamental task, particularly in constantly changing situations. Although deep reinforcement learning (DRL) techniques exist that utilize the positional information of robot's, environmental states, and input dataset for neural networks. Although, the positional information alone does not provide sufficient insights into the motion trends of obstacles. To solve this issue, this paper presents a dynamic obstacle mobility pattern approach for mobile robots (MRs) that rely on DRL. This method employs the positional details of dynamic obstacles dependent upon time for establishing a movement trend vector. This vector, in conjunction with another mobility state attribute, forms the MR mobility guidance matrix, that essentially conveys the pattern variation of dynamic obstacles trend over a specified interval. Using this matrix, the robot can choose its avoidance action. Also, this methodology uses the DRL-based dynamic policy algorithm for the testing and validation of the proposed technique through Python programming. The experimental outcomes demonstrate that this technique substantially improves the safety of avoiding dynamic obstacles.
Position control of a mobile robot using reinforcement learning
IFAC-PapersOnLine, 2020
Robotics has been introduced in education at all levels during the last years. In particular, the application of mobile robots for teaching automatic control is becoming more popular in engineering because of the attractive experiments that can be performed. This paper presents the design, development, and implementation of an algorithm to control the position of a wheeled mobile robot using Reinforcement Learning in an advanced 3D simulation environment. In this approach, the learning process occurs when the agent makes some actions in the environment to get some rewards. Trying to make a balance between the new information of the environment and the current knowledge about it. In this way, the algorithm is divided into two phases: 1) the learning stage, and 2) the operational stage. In the first stage, the robot learns how to reach a known destination point from its current position. To do it, it uses the information of the environment and the rewards, to build a learning matrix that is used later during the operational stage. The main advantage of this algorithm concerning traditional control algorithms is that the learning process is carried out automatically with a recursive procedure and the result is a controller that can make the specific task, without the need for a dynamic model. Its main drawback is that the learning stage can take a long time to finish and it depends on the hardware resources of the computer used during the learning process.
A Novel Behavioral Strategy for RoboCode Platform Based on Deep Q-Learning
Complexity
This paper addresses a new machine learning-based behavioral strategy using the deep Q-learning algorithm for the RoboCode simulation platform. According to this strategy, a new model is proposed for the RoboCode platform, providing an environment for simulated robots that can be programmed to battle against other robots. Compared to Atari Games, RoboCode has a fairly wide set of actions and situations. Due to the challenges of training a CNN model for such a continuous action space problem, the inputs obtained from the simulation environment were generated dynamically, and the proposed model was trained by using these inputs. The trained model battled against the predefined rival robots of the environment (standard robots) by cumulatively benefiting from the experience of these robots. The comparison between the proposed model and standard robots of RoboCode Platform was statistically verified. Finally, the performance of the proposed model was compared with machine learning based-...