How to teach a machine to drive in difficult situations and be able to rely on it

Autonomous driving is an active field of research in academia and industry. On the way to the ambitious goal of fully autonomous driving, many in the development of Advanced Driver Assistance Systems (ADASs) address the problem of driving assistance in difficult situations. We embarked on the adventure of using reinforcement learning to design such a system. The central idea of Reinforcement Learning is to use a reward function that enables the artificial intelligence (AI) to learn a good rule of behavior on its own by interaction with the environment. The result is a policy for the autonomous agent. Along the way, we discovered that it is prudent not to use this advanced method to learn behavior in a so-called end-to-end scenario, i.e. from image/video data to actuators such as steering and breaks. It is better placed in the middle of the driving process by using abstract representations inferred from the sensors and learning to decide on more high-level actions or objectives, e.g., whether to perform a lane-change or not. This reduces the complexity of the task and at the same time enables us to understand and verify the behavior of the learned agent. Another challenge rises from the fact that many situations like collisions of cars can only be explored in simulation and cannot be learned experimentally with actual vehicles. Our work shows that a tailored randomization strategy vastly increases the robustness of the learned agent towards variations in the scenario it is employed in.