Application of AI based reinforcement learning to robot vehicle control (original) (raw)
Related papers
ICCSA 2024, 2024
Currently, autonomous cars are extensively studied by many institutions and companies with the aim of practical implementation. Q-learning is an algorithm of reinforcement learning, which does not require a model and can be seen as an asynchronous method of dynamic programming. It allows agents to learn to take action optimally in a Markovian environment by experiencing the outcomes of actions without the need to build a probability model. This paper presents the process of developing an autonomous vehicle model for educational purposes that can self-optimize its decision-making options based on a reinforcement learning algorithm such as Q-learning. The algorithm helps train autonomous cars to avoid obstacles collisions. The preliminary results show that the Q-learning algorithm is successful in building a self-training technique to adapt to specific requirements.
Conference on Developments in Computer Science Budapest, Hungary, June 17-19, 2021, 2021
With the continuous improvement of global economic integration, the pace of global urbanization and modernization has been accelerated. And with the agricultural and industrial revolutions, the urban population began to expand at an unprecedented rate. Today, 55% of the world's population lives in urban areas, a proportion that is expected to increase to 68% by 2050, additional 2.5 billion city dwellers worldwide. Along with the physical expansion of cities, there are many problems, such as water management problems, social order problems, traffic problems. Among them like the blood of the city traffic, more and more become an obstacle to urban development. Solving the problem of urban traffic congestion and optimize the efficiency of urban traffic is an essential link in building a smart city. Intelligent transportation is an important part of a smart city. How to scientifically design and build intelligent transportation is a very valuable problem. Reinforcement learning, as an important part of the Computer Science, has been widely used in various social fields such as Health, transportation, education, finance, and so on. This paper will discuss the application of intelligent transportation control systems through the investigation and research of intelligent transportation combined with reinforcement learning.
This paper presents a self-improving reactive control system for autonomous robotic navigation. The navigation module uses a schemabased reactive control system to perform the navigation task. The learning module combines case-based reasoning and reinforcement learning to continuously tune the navigation system through experience. The case-based reasoning component perceives and characterizes the system's environment, retrieves an appropriate case, and uses the recommendations of the case to tune the parameters of the reactive control system. The reinforcement learning component refines the content of the cases based on the current experience. Together, the learning components perform on-line adaptation, resulting in improved performance as the reactive control system tunes itself to the environment, as well as on-line learning, resulting in an improved library of cases that capture environmental regularities necessary to perform on-line adaptation. The system is extensively evaluated through simulation studies using several performance metrics and system configurations.
A Method of Deep Reinforcement Learning for Simulation of Autonomous Vehicle Control
2021
Nowadays autonomous driving is expected to revolutionize the transportation sector. Carmakers, researchers, and administrators have been working on this field for years and significant progress has been made. However, the doubts and challenges to overcome are still huge, regarding not only complex technologies but also human awareness, culture, current traffic infrastructure. In terms of technical perspective, the accurate detection of obstacles, avoiding adjacent obstacles, and automatic navigation through the environment are some of the difficult problems. In this paper, an approach for solving those problems is proposed by using of Policy Gradient to control a simulated car via reinforcement learning. The proposed method is worked effectively to train an agent to control the simulated car in Unity ML-agents Highway, which is a simulating environment. This environment is chosen from some criteria of an environment simulating autonomous vehicle. The testing of the proposed method got positive results. Beside the average speed was well, the agent successfully learned the turning operation, progressively gaining the ability to navigate larger sections of the simulated raceway without crashing.
Reinforcement learning for the control of traffic flow on highways
2018
By submitting this dissertation electronically, I declare that the entirety of the work contained therein is my own, original work, that I am the sole author thereof (save to the extent explicitly otherwise stated), that reproduction and publication thereof by Stellenbosch University will not infringe any third party rights and that I have not previously in its entirety or in part submitted it for obtaining any qualification.
Self-Navigation Car using Reinforcement Learning
2019
In this paper, a project is described which is a 2-D modelled version of a car that will learn how to drive itself. It will have to figure everything out on its own. In addition, to achieve that the simulator contains a car running simultaneously &can be controlled by different control algorithms - heuristic, reinforcement learning-based, etc. For each dynamic input, the Reinforcement- Learning modifies new patterns. Ultimately, Reinforcement Learning helps in maximizing the reward from every state. In this first Part, we will implement a Reinforcement-Learning model to build an AI for Self Driving Car. Project will be focusing on the brain of the car not any graphics. The car will detect obstacles and take basic actions. To make autonomous car or self-driving car a reality, some of the factors to be considered are human safety and quality of life.
An ML-Aided Reinforcement Learning Approach for Challenging Vehicle Maneuvers
IEEE Transactions on Intelligent Vehicles
The richness of information generated by today's vehicles fosters the development of data-driven decision-making models, with the additional capability to account for the context in which vehicles operate. In this work, we focus on Adaptive Cruise Control (ACC) in the case of such challenging vehicle maneuvers as cut-in and cutout , and leverages Deep Reinforcement Learning (DRL) and vehicle connectivity to develop a data-driven cooperative ACC application. Our DRL framework accounts for all the relevant factors, namely, passengers' safety and comfort as well as efficient road capacity usage, and it properly weights them through a two-layer learning approach. We evaluate and compare the performance of the proposed scheme against existing alternatives through the CoMoVe framework, which realistically represents vehicle dynamics, communication and traffic. The results, obtained in different real-world scenarios, show that our solution provides excellent vehicle stability, passengers' comfort, and traffic efficiency, and highlight the crucial role that vehicle connectivity can play in ACC. Notably, our DRL scheme improves the road usage efficiency by being inside the desired range of headway in cutout and cut-in scenarios for 69% and 78% (resp.) of the time, whereas alternatives respect the desired range only for 15% and 45% (resp.) of the time. We also validate the proposed solution through a hardware-in-the-loop implementation, and demonstrate that it achieves similar performance to that obtained through the CoMoVe framework. Index Terms-Machine learning-based vehicle applications, connected vehicles, vehicle dynamics, adaptive cruise control.
Highway Environment Model for Reinforcement Learning
IFAC-PapersOnLine, 2018
The paper presents a microscopic highway simulation model, built as an environment for the development of different machine learning based autonomous vehicle controllers. The environment is based on the popular OpenAI Gym framework, hence it can be easily integrated into multiple projects. The traffic flow is operated by classic microscopic models, while the agent's vehicle uses a rigid kinematic single-track model, with either continuous or discrete action spaces. The environment also provides a simple high-level sensor model, where the state of the agent and its surroundings are part of the observation. To aid the learning process, multiple reward functions are also provided.
Designs
Many classic control approaches have already proved their merits in the automotive industry. Model predictive control (MPC) is one of the most commonly used methods. However, its efficiency drops off with increase in complexity of the driving environment. Recently, machine learning methods have been considered an efficient alternative to classical control approaches. Even with successful implementation of reinforcement learning in real-world applications, it is still not commonly used compared to supervised and unsupervised learning. In this paper, a reinforcement learning (RL)-based framework is suggested for application in autonomous driving systems to maintain a safe distance. Additionally, an MPC-based control model is designed for the same task. The behavior of the two controllers is compared and discussed. The trained RL model was deployed on a low-end FPGA-in-the-loop (field-programmable gate array in-the-loop). The results showed that the two controllers responded efficientl...
Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
In this work, we propose a hierarchical reinforcement learning (HRL) structure which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-goals simultaneously; 2) extracting attentions of states according to changing sub-goals during the learning process; 3) reusing the well-trained network of sub-goals for other similar tasks with the same sub-goals. The states are defined as processed observations which are transmitted from the perception system of the autonomous vehicle. A hybrid reward mechanism is designed for different hierarchical layers in the proposed HRL structure. Compared to traditional RL methods, our algorithm is more sample-efficient since its modular design allows reusing the policies of sub-goals across similar tasks. The results show that the proposed method converges to an optimal policy faster than traditional RL methods.