GIFT Team Publishes Paper at IROS, a Premier Flagship Conference on Robotics

The Intelligent Centre of Intelligent Connected Electric Vehicles of Shanghai Jiao Tong University, under the guidance of Professors Qin Tong and Yang Ming, has published a research paper titled "Embodied Escaping: End-to-End Reinforcement Learning for Robot Navigation in Narrow Environments" at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), a flagship conference in the field of robotics. The study addresses the autonomous escape problem of cleaning robots in confined spaces, providing a systematic solution for end-to-end robot navigation. The co-first authors of the paper are Zheng Han, a 2025 PhD candidate at GIFT, and Zhang Jiale, a 2023 PhD candidate at the School of Automation and Intelligent Sensing.

Research Background

In household cleaning scenarios, it is inevitable for cleaning robots to enter narrow dead zones in order to perform full-coverage tasks. However, traditional methods remain ineffective due to the following reasons:

1)Highly non-convex environments and intensive obstacles drastically reduce free space, making geometric approximations ineffective at maintaining safety margins and significantly increasing collision risks.

2)Navigation relies on precise environmental modeling, where even minor map errors can lead to planning failures.

3)Non-holistic dynamic constraints render feasible escape trajectories extremely sparse in high-dimensional continuous spaces.

4)Escaping confined spaces often requires complex motivations, including frequent directional changes and saturated turning rates, resulting in trajectories filled with sharp turns and abrupt transitions.

These coupled challenges cause existing reinforcement learning and approaches to fail in dead-zone scenarios, and call for novel solutions.

Research Results

To comprehensively address issues above, the paper proposes an end-to-end mapless escape framework based on reinforcement learning. To mitigate the sparse rewards related to long-distance navigation, the paper adopts a hybrid training strategy, which leverages the online A* algorithm and reverse-calculated action sequences to accelerate convergence near target points. To handle redundant and ineffective actions, the study spearheads a new action representation, reshaping the discrete action space by proportionally scaling linear and angular velocities with a unified turning radius. Additionally, an improved action masking method is adopted to rapidly select valid actions and provide neural networks with prior features of effective action distributions, balancing precision and efficiency.

Framework diagram of the embodied escape model

Extensive comparative simulations demonstrate that the proposed method outperforms traditional methods and reinforcement learning approaches in terms of success rate and collision avoidance. In real-world experiments, robots equipped with lidar, IMU and dual-wheel encoders can successfully escape narrow and complex dead zones in experiments of varying difficulties, validating the model's generalization capability in practical scenarios.

Visual comparison of escape performance across algorithms

Method Showcase and Real-World Experiment

Author Information

Zheng Han

PhD candidate (Class of 2025) at GIobal Institute of Future Technology, SJTU. Research interests: General robot learning, end-to-end autonomous driving and generative artificial intelligence.

Zhang Jiale

PhD candidate (Class of 2023) at the School of Automation and Intelligent Sensing, SJTU. Research interests: End-to-end autonomous driving and vehicle following.

Jiang Mingyang

Master's candidate (Class of 2023) at the School of Sensing Science and Engineering, SJTU. Research interests: Autonomous driving decision-making and motion planning based on reinforcement learning.

Qin Tong

Associate Professor at Global Institute of Future Technology, SJTU. Prof Qin earned his PhD degree from the Department of Electronic and Computer Engineering at the Hong Kong University of Science and Technology and previously worked at Intelligent Automotive Solution BU of Huawei. Qin was selected as one of Huawei's "Genius Youth." During his tenure as a perception and SLAM expert at Huawei, he contributed to the development of the Huawei ADS autonomous driving system, delivering lead-edge solutions that have been commercially deployed in multiple vehicle models. In recent years, he has published over ten high-quality papers as first/corresponding author in top-tier robotics journals and conferences such as TRO, JFR, RAL and ICRA. He received the IROS 2018 Best Student Paper Award and a TRO Best Paper Nomination Award. Research interests: Autonomous driving perception, mapping and localization; end-to-end AI large models; mobile robot SLAM.

Yang Ming

Distinguished Professor at the School of Sensing Science and Engineering, SJTU, PhD supervisor, and Director of the Intelligent Centre of Intelligent Connected Electric Vehicles. He is a leading talent in the National "Ten Thousand Talents Plan" for scientific and technological innovation. Currently, he serves as Deputy Director of the Intelligent Vehicle Committee and the Education Committee of the Chinese Association of Automation, Council Member of the Chinese Association for Artificial Intelligence, Deputy Director of Intelligent Robotics Committee and Associate Editor of IEEE IROS-CPRB. He is also an editorial board member for IEEE Transactions on Intelligent Vehicles and IEEE Transactions on Intelligent Transportation Systems. With long-standing expertise in unmanned vehicles and intelligent robotics, Prof Yang has published over 200 academic papers and holds more than 50 authorized national patents. His students have won outstanding paper awards at top-tier international conferences such as IEEE Intelligent Vehicle Symposium and the China Conference on Intelligent Robotics. He has received the Shanghai Technology Invention First Prize, Shanghai Teaching Achievement First Prize and the Ministry of Education Technology Invention Award, among other accolades. Research interests: Low-speed unmanned driving systems.