学术讲座：Synthesis and Evaluation of Automated Vehicles' Decision-making Systems
主讲人：Songan Zhang, Research Scientist, Robotics Research Team, Ford Motor Company
时间：9:00 - 10:00, May 25, 2023 (Beijing Time)
地点：Room 202, Yue-Kong Pao Library's Annex
讲座摘要
This presentation focuses on the synthesis of a decision-making system for Automated Vehicles (AVs), and then evaluates the safety of the system with an eye toward improving the design. We begin with a synthesis of an AV's decision-making system in a specific driving environment. We model the environment as a Markov Decision Process (MDP), with the goal of determining the optimal strategy (that is, policy) for this particular MDP. We propose a novel Reinforcement Learning (RL) method using model-based exploration. This method allows the training agent to explore the MDP state space by maximizing the notion of an agent’s surprise about its experiences via intrinsic motivation. The optimal strategy will be deemed to be a global-optimal policy by which the AV can travel more efficiently. Then we introduce two safety evaluation methods. One is based on naturalistic driving data, and the other one designed an attacker which is capable of generating socially acceptable attacks. The crash rate of the system then becomes 50 times greater in the environment with the attacker, which allows the system to register fatal flaws in the original training environment design. Therefore, our next goal is to improve the original policy so as to design a safe and robust decision-making system under situations with different types of drivers in the environment (including previously designed attacker), different traffic densities, and differing numbers of total surrounding vehicles. We tackle this problem by implementing the state-of-the-art Meta-Reinforcement Learning (Meta RL) method to train an agent to quickly adapt to different environments with limited data. The Meta RL-trained policy can significantly decrease the crash rate with a small amount of data across different environments. This technique has tremendous potential for helping the AV quickly adapt to varying conditions such as different locations, weather, and lighting.
主讲人简介
Songan Zhang received B.S. and M.S. degrees in automotive engineering from Tsinghua University in 2013 and 2016, respectively. Then she went to the University of Michigan, Ann Arbor, and got a Ph.D. degree in mechanical engineering in 2021. She is currently working at Ford Motor Company in the Robotics Research Team. Her research interests include accelerated evaluation of autonomous vehicles and reinforcement learning and meta-reinforcement learning for autonomous vehicle decision-making systems.
