Introduction

Professional race car drivers can execute extreme overtaking maneuvers. However, conventional systems for autonomous overtaking rely on either simplified assumptions about the vehicle dynamics or solving expensive trajectory optimization problems online. When the vehicle is approaching its physical limits, existing model-based controllers struggled to handle highly nonlinear dynamics and cannot leverage the large volume of data generated by simulation or real-world driving. To circumvent these limitations, this work proposes a new learning-based method to tackle the autonomous overtaking problem. We evaluate our approach using Gran Turismo Sport---a world-leading car racing simulator known for its detailed dynamic modeling of various cars and tracks. By leveraging curriculum learning, our approach leads to faster convergence as well as increased performance compared to vanilla reinforcement learning. As a result, the trained controller outperforms the built-in model-based game AI and achieves comparable overtaking performance with an experienced human driver.

Video

Reference

          
@article{song2021autonomous,
  title={Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning},
  author={Song, Yunlong and Lin, HaoChih and Kaufmann, Elia and Duerr, Peter and Scaramuzza, Davide},
  booktitle={International Conference on Robotics and Automation (ICRA)},
  year={2021},
  organization={IEEE}
}

Introduction

Autonomous car racing is a major challenge in robotics. It raises fundamental problems for classical approaches such as planning minimum-time trajectories under uncertain dynamics and controlling the car at its limits of handling. Besides, the requirement of minimizing the lap time, which is a sparse objective, and the difficulty of collecting training data from human experts have also hindered researchers from directly applying learning-based approaches to solve the problem. In the present work, we propose a learning-based system for autonomous car racing by leveraging high-fidelity physical car simulation, a course-progress-proxy reward, and deep reinforcement learning. We deploy our system in Gran Turismo Sport, a world-leading car simulator known for its realistic physics simulation of different race cars and tracks, which is even used to recruit human race car drivers. Our trained policy achieves autonomous racing performance that goes beyond what had been achieved so far by the built-in AI, and, at the same time, outperforms the fastest driver in a dataset of over 50,000 human players.

Video

Reference

          
@article{fuchs2021super,
  title={Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning},
  author={Fuchs, Florian and Song, Yunlong and Kaufmann, Elia and Scaramuzza, Davide and Duerr, Peter},
  journal={IEEE Robotics and Automation Letters}, 
  year={2021},
  publisher={IEEE}
}

Fast and Furious II

Introduction

Video

Reference

Fast and Furious I

Introduction

Video

Reference