Autonomous driving involves complex decision-making in highly interactive environments, requiring thoughtful negotiation with other traffic participants. While reinforcement learning provides a way to learn such interaction behavior, efficient learning critically depends on scalable state representations. Contrary to imitation learning methods, high-dimensional state representations still constitute a major bottleneck for deep reinforcement learning methods in autonomous driving. In this paper, we study the challenges of constructing bird's-eye-view representations for autonomous driving and propose a recurrent learning architecture for long-horizon driving. Our PPO-based approach, called RecurrDriveNet, is demonstrated on a simulated autonomous driving task in CARLA, where it outperforms traditional frame-stacking methods while only requiring one million experiences for training. RecurrDriveNet causes less than one infraction per driven kilometer by interacting safely with other road users.
Current reinforcement learning approaches for learning driving policies face the bottleneck of dimensionality.
In this paper, we evaluate the efficiency of various bird's-eye-view (BEV) representations used for describing the state of the driving scene.
In addition to that, we propose a novel LSTM-based encoding scheme for efficiently encoding the bird's-eye-view state representation across the full trajectory of states in a RL fashion.
This alleviates the need for old-fashioned frame-stacking methods and enables further long-horizon driving research.
Based on our chosen LSTM-based encoding of BEVs we achieve significantly higher average returns while reducing the number of infractions when driving compared to frame-stacking methods. This allows, e.g., also robust stopping at red traffic lights.
If you find our findings useful, please consider citing our paper:
@article{trumpp2023efficient,
title={Efficient Learning of Urban Driving Policies Using Bird's-Eye-View State Representations},
author={Raphael Trumpp and Martin Büchner and Abhinav Valada and Marco Caccamo},
year={2023},
eprint={2305.19904},
archivePrefix={arXiv},
primaryClass={cs.RO}
}
