Simulation: Development of a Data-Driven Optimal Controller Based on Adaptive Dynamic Programming

This is the SUMO simulation presented in the report: Development of a Data-Driven Optimal Controller Based on Adaptive Dynamic Programming. Through vehicle-to-vehicle (V2V) communication, both human-driven and autonomous vehicles can actively exchange data, such as velocities and bumper-to-bumper distances. Employing By employing the shared data, control laws with improved performance can be designed for connected and autonomous vehicles (CAVs). In this report, while taking into account human-vehicle interaction and heterogeneous driver behavior, an adaptive optimal control design method is proposed for a platoon mixed with multiple preceding human-driven vehicles and one CAV at the tail. It is shown that, by using reinforcement-based learning and adaptive dynamic programming techniques, a near-optimal controller can be learned from real-time data for the CAV with V2V communications, but and do so without the precise knowledge of the accurate car-following parameters of any driver in the platoon. The proposed method allows the CAV controller to adapt to different platoon dynamics caused by the unknown and heterogeneous driver-dependent parameters. To improve safety performance during the learning process, our off-policy learning algorithm can leverage both the historical data and the data collected in real-time, which leads to considerably reduced learning time duration. The effectiveness and efficiency of our proposed method are demonstrated by rigorous proofs and microscopic traffic simulations.