Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection (original) (raw)
Related papers
ArXiv, 2021
Prior research has extensively explored Autonomous Vehicle (AV) navigation in the presence of other vehicles, however, navigation among pedestrians, who are the most vulnerable element in urban environments, has been less examined. This paper explores AV navigation in crowded, unsignalized intersections. We compare the performance of different deep reinforcement learning methods trained on our reward function and state representation. The performance of these methods and a standard rule-based approach were evaluated in two ways, first at the unsignalized intersection on which the methods were trained, and secondly at an unknown unsignalized intersection with a different topology. For both scenarios, the rule-based method achieves less than 40% collision-free episodes, whereas our methods result in a performance of approximately 100%. Of the three methods used, DDQN/PER outperforms the other two methods while it also shows the smallest average intersection crossing time, the greatest...
Unexpected Collision Avoidance Driving Strategy Using Deep Reinforcement Learning
IEEE Access, 2020
In this paper, we generated intelligent self-driving policies that minimize the injury severity in unexpected traffic signal violation scenarios at an intersection using the deep reinforcement learning. We provided guidance on reward engineering in terms of the multiplicity of objective function. We used a deep deterministic policy gradient method in the simulated environment to train self-driving agents. We designed two agents, one with a single-objective reward function of collision avoidance and the other with a multi-objective reward function of both collision avoidance and goal-approaching. We evaluated their performances by comparing the percentages of collision avoidance and the average injury severity against those of human drivers and an autonomous emergency braking (AEB) system. The percentage of collision avoidance of our agents were 78.89% higher than human drivers and 84.70% higher than the AEB system. The average injury severity score of our agents were only 8.92% of human drivers and 6.25% of the AEB system.
End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning
2021 IEEE Intelligent Vehicles Symposium (IV), 2021
Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. However, for the majority of intersections regulated by traffic lights, the problem could be solved by a simple rule-based method in which the autonomous vehicle behavior is closely related to the traffic light states. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step. We demonstrate that agents learn both the basic rules needed to handle intersections by understanding the priorities of other learners inside the environment, and to drive safely along their paths. Moreover, a comparison between our system and a rulebased method proves that our model achieves better results especially with dense traffic conditions. Finally, we test our system on real world scenarios using real recorded traffic data, proving that our module is able to generalize both to unseen environments and to different traffic conditions.
2020 IEEE Intelligent Vehicles Symposium (IV)
Automated lane change is one of the most challenging task to be solved of highly automated vehicles due to its safety-critical, uncertain and multi-agent nature. This paper presents the novel deployment of the state of art Q learning method, namely Rainbow DQN, that uses a new safety driven rewarding scheme to tackle the issues in an dynamic and uncertain simulation environment. We present various comparative results to show that our novel approach of having reward feedback from the safety layer dramatically increases both the agent's performance and sample efficiency. Furthermore, through the novel deployment of Rainbow DQN, it is shown that more intuition about the agent's actions is extracted by examining the distributions of generated Q values of the agents. The proposed algorithm shows superior performance to the baseline algorithm in the challenging scenarios with only 200000 training steps (i.e. equivalent to 55 hours driving).
Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions
IEEE Transactions on Intelligent Transportation Systems, 2021
Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-theart strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.
Autonomous Driving using Deep Reinforcement Learning in Urban Environment
Hashim Shakil Ansari | Goutam R, 2019
Deep Reinforcement Learning has led us to newer possibilities in solving complex control and navigation related tasks. The paper presents Deep Reinforcement Learning autonomous navigation and obstacle avoidance of self driving cars, applied with Deep Q Network to a simulated car an urban environment. “The car, using a variety of sensors will be easily able to detect pedestrians, objects will allow the car to slow or stop suddenly. As a computer is far more precise and subject to fewer errors than a human, accident rates may reduce when these vehicles become available to consumers. This autonomous technology would lead to fewer traffic jams and safe roadâ€. Hashim Shakil Ansari | Goutam R ""Autonomous Driving using Deep Reinforcement Learning in Urban Environment"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-3 , April 2019, URL: https://www.ijtsrd.com/papers/ijtsrd23442.pdf Paper...
Deep adaptive learning for safe and efficient navigation of pedestrian dynamics
IET Intelligent Transport Systems, 2021
An efficient and safe evacuation of passengers is important during emergencies. Overcapacity on a route can cause an increased evacuation time. Decision making is essential to optimally guide and distribute pedestrians to multiple routes while ensuring safety. Developing an optimal pedestrian path planning route while considering learning dynamics and uncertainties in the environment generated from pedestrian behaviour is challenging. While previous evacuation planning studies have focused on either simulation of realistic behaviours or simple route planning, the best route decisions with several intermediate decision-points, especially under real-time changing environments, have not been considered. This paper develops an optimal navigation model providing more navigation guidance for evacuation emergencies to minimize the total evacuation time while considering the influence of other passengers based on the social-force model. The integration of the optimal navigation model was ultimately able to reduce the overall evacuation time of multiple scenarios presented with two different overall pedestrian totals. The overall maximum evacuation time savings presented was 10.6%.
Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
Sensors
Intersections are considered one of the most complex scenarios in a self-driving framework due to the uncertainty in the behaviors of surrounding vehicles and the different types of scenarios that can be found. To deal with this problem, we provide a Deep Reinforcement Learning approach for intersection handling, which is combined with Curriculum Learning to improve the training process. The state space is defined by two vectors, containing adversaries and ego vehicle information. We define a features extractor module and an actor–critic approach combined with Curriculum Learning techniques, adding complexity to the environment by increasing the number of vehicles. In order to address a complete autonomous driving system, a hybrid architecture is proposed. The operative level generates the driving commands, the strategy level defines the trajectory and the tactical level executes the high-level decisions. This high-level decision system is the main goal of this research. To address ...
Deep-Q Learning for Autonomous Driving System in Simulation Environment
2020
Abstract: Due to increase in traffic accidents by negligent driving by human beings, there is a need for safe and comfortable driving experience. For this purpose, development of a simulation for autonomous driving car which later can be developed into a physical model after the simulation training has been successful. This project deals with simulation results of an autonomous car learning to drive on a single-lane highway or a road with lane markings. Learning is performed by using Deep Q algorithm. For a given input image of the street captured by the camera which is installed at the front of the car, the Deep Q Network computes the Q values (rewards) corresponding to the actions available to the autonomous driving car. These actions are discrete angles through which the car can steer for constant speed. The autonomous driving system in the car enforces action that has the highest reward. Index Terms – Artificial Intelligence, AI, Self-driving Cars, Autonomous Driving cars, Reinf...