Published March 31, 2024 | Version v1
Journal article Open

Safety-Eye: On Road Safety for Self-driving Vehicles using Reinforcement Learning

Authors/Creators

Description

With technological advancements automotive industry is witnessing new era of transition to self-driving vehicle for everything. Self-driving cars for personal and commercial use are right across the corner to become full reality. Along with self-driving drone and robots for delivery of food and merchandise have become reality.  With these advancements the safety and security of the self-driving vehicles have emerged as major concern for current times. To this end, this work proposes SafeEye a reinforcement learning based policy aware and optimized approach with federated learning to provide self-driving vehicles better and safer route map and runtime optimization. The simulation results of SafeEye based system with multiple scenarios shows promising results and opens a new research direction to explore to provide safe and informed route prediction for self-driving vehicles of next generation.

Files

EJAET-11-3-131-135.pdf

Files (291.7 kB)

Name Size
md5:4d7e5369fb092f7eded7b7ad8edd2f3d
291.7 kB Preview Download

Additional details

References

  • [1]. E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda, "A survey of autonomous driving: common practices and emerging technologies," arXiv preprint arXiv:1906.05113, 2019.
  • [2]. V. Talpaert, I. Sobh, B. R. Kiran, P. Mannion, S. Yogamani, A. ElSallab, and P. Perez, "Exploring applications of deep reinforcement learning for real-world autonomous driving systems," arXiv preprint arXiv:1901.01536, 2019.
  • [3]. B. R. Kiran, I. Sobh, V. Talpaert, P. Mannion, A. A. A. Sallab, S. Yogamani, and P. Perez, "Deep reinforcement learning for autonomous ´ driving: A survey," arXiv preprint arXiv:2002.00444, 2020
  • [4]. W. Schwarting, J. Alonso-Mora, and D. Rus, "Planning and decisionmaking for autonomous vehicles," Annual Review of Control, Robotics, and Autonomous Systems, 2018.
  • [5]. D. Gonzalez, J. P´erez, V. Milan`es, and F. Nashashibi, "A review of motion planning techniques for automated vehicles," IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 4, pp. 1135–1145, 2015.
  • [6]. B. Paden, M. Cˇ ap, S. Z. Yong, D. Yershov, and E. Frazzoli, "A ´ survey of motion planning and control techniques for self-driving urban vehicles," IEEE Transactions on intelligent vehicles, vol. 1, no. 1, pp. 33–55, 2016.
  • [7]. F. Ye, S. Zhang, P. Wang and C. -Y. Chan, "A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles," 2021 IEEE Intelligent Vehicles Symposium (IV), Nagoya, Japan, 2021, pp. 1073-1080, doi: 10.1109/IV48863.2021.9575880.
  • [8]. Shubbak, M. (2013, January 1). Self-Driving Cars: Legal, Social, and Ethical Aspects. RELX Group (Netherlands). https://doi.org/10.2139/ssrn.2931847
  • [9]. Rasouli, A., & Tsotsos, J K. (2018, January 1). Joint Attention in Driver-Pedestrian Interaction: from Theory to Practice. Cornell University. https://doi.org/10.48550/arxiv.1802.02522
  • [10]. SFO crowd sets self-driving car on fire https://www.washingtonpost.com/technology/2024/02/12/waymo-set-on-fire-san-francisco/
  • [11]. Robbing delivery robots is now a thing https://www.autoweek.com/news/a44839987/delivery-robots-being-robbed/
  • [12]. Porch pirates https://www.ktalnews.com/news/consumer-reports/how-ai-and-drones-will-help-prevent-porch-pirates-after-packages-arrive/
  • [13]. self-driving car blocked ambulance https://ktla.com/news/california/company-refutes-claim-that-self-driving-cars-blocked-california-ambulance-led-to-victims-death/#:~:text=According%20to%20the%20San%20Francisco,crews%20from%20reaching%20the%20hospital.
  • [14]. Jakob N Foerster, Yannis M Assael, Nando De Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. arXiv preprint arXiv:1605.06676, 2016.
  • [15]. Simon Vanneste, Astrid Vanneste, Siegfried Mercelis, and Peter Hellinckx. Learning to communicate using counterfactual reasoning. arXiv preprint arXiv:2006.07200, 2020.
  • [16]. Yali Du, Lei Han, Meng Fang, Ji Liu, Tianhong Dai, and Dacheng Tao. Liir: Learning individual intrinsic reward in multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 32:4403–4414, 2019.
  • [17]. Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  • [18]. Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pages 4295–4304. PMLR, 2018.
  • [19]. Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350–354, 2019.
  • [20]. Johannes Heinrich and David Silver. Deep reinforcement learning from self-play in imperfect-information games. arXiv preprint arXiv:1603.01121, 2016.
  • [21]. Joel Z Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. Multi-agent reinforcement learning in sequential social dilemmas. arXiv preprint arXiv:1702.03037, 2017.
  • [22]. Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 2017.
  • [23]. Wilko Schwarting, Alyssa Pierson, Javier Alonso-Mora, Sertac Karaman, and Daniela Rus. Social behavior for autonomous vehicles. Proceedings of the National Academy of Sciences, 116(50):24972–24978, 2019
  • [24]. Behrisch, Michael, et al. "SUMO–simulation of urban mobility: an overview." Proceedings of SIMUL 2011, The Third International Conference on Advances in System Simulation. ThinkMind, 2011.
  • [25]. Dosovitskiy, Alexey, et al. "CARLA: An open urban driving simulator." Conference on robot learning. PMLR, 2017.
  • [26]. Tang, Zheng, et al. "Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
  • [27]. Cathy Wu, Aboudy Kreidieh, Kanaad Parvate, Eugene Vinitsky, and Alexandre M Bayen. Flow: Architecture and benchmarking for reinforcement learning in traffic control. arXiv preprint arXiv:1710.05465, page 10, 2017.
  • [28]. Ming Zhou, Jun Luo, Julian Villella, Yaodong Yang, David Rusu, Jiayu Miao, Weinan Zhang, Montgomery Alban, Iman Fadakar, Zheng Chen, Aurora Chongxi Huang, Ying Wen, Kimia Hassanzadeh, Daniel Graves, Dong Chen, Zhengbang Zhu, Nhat Nguyen, Mohamed Elsayed, Kun Shao, Sanjeevan Ahilan, Baokuan Zhang, Jiannan Wu, Zhengang Fu, Kasra Rezaee, Peyman Yadmellat, Mohsen Rohani, Nicolas Perez Nieves, Yihan Ni, Seyedershad Banijamali, Alexander Cowen Rivers, Zheng Tian, Daniel Palenicek, Haitham bou Ammar, Hongbo Zhang, Wulong Liu, Jianye Hao, and Jun Wang. Smarts: Scalable multiagent reinforcement learning training school for autonomous driving, 2020.