Publications

2026

1. arXiv

   

   Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving

   Zilin Huang, Zhengyang Wan, Zihao Sheng, and 4 more authors

   arXiv preprint arXiv:2604.03497 · 2026

   Abs Paper

   Scholar 0

   Deploying reinforcement learning policies trained in simulation to real autonomous vehicles remains a fundamental challenge, particularly for VLM-guided RL frameworks whose policies are typically learned with simulatornative observations and simulator-coupled action semantics that are unavailable on physical platforms. This paper presents Sim2Real-AD, a modular framework for zero-shot sim-to-real transfer of CARLA-trained VLM-guided RL policies to full-scale vehicles without any real-world RL training data.

   @article{huang2026sim2real,
     title = {Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving},
     author = {Huang, Zilin and Wan, Zhengyang and Sheng, Zihao and Wang, Boyue and You, Junwei and Leng, Yue and Chen✉, Sikai},
     journal = {arXiv preprint arXiv:2604.03497},
     year = {2026},
     priority = {0},
   }

2. arXiv

   

   DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving

   Zilin Huang, Zihao Sheng, Zhengyang Wan, and 4 more authors

   arXiv preprint arXiv:2603.18315 · 2026

   Abs Paper

   Scholar 2

   Ensuring safe decision-making in autonomous vehicles remains a fundamental challenge despite rapid advances in end-to-end learning approaches. Traditional reinforcement learning (RL) methods rely on manually engineered rewards or sparse collision signals, which fail to capture the rich contextual understanding required for safe driving and make unsafe exploration unavoidable in real-world settings. Recent vision-language models (VLMs) offer promising semantic understanding capabilities; however, their high inference latency and susceptibility to hallucination hinder direct application to real-time vehicle control. To address these limitations, this paper proposes DriveVLM-RL, a neuroscience-inspired framework that integrates VLMs into RL through a dual-pathway architecture for safe and deployable autonomous driving. The framework decomposes semantic reward learning into a Static Pathway for continuous spatial safety assessment using CLIP-based contrasting language goals, and a Dynamic Pathway for attention-gated multi-frame semantic risk reasoning using a lightweight detector and a large VLM. A hierarchical reward synthesis mechanism fuses semantic signals with vehicle states, while an asynchronous training pipeline decouples expensive VLM inference from environment interaction. All VLM components are used only during offline training and are removed at deployment, ensuring real-time feasibility. Experiments in the CARLA simulator show significant improvements in collision avoidance, task success, and generalization across diverse traffic scenarios, including strong robustness under settings without explicit collision penalties. These results demonstrate that DriveVLM-RL provides a practical paradigm for integrating foundation models into autonomous driving without compromising real-time feasibility.

   @article{huang2026drivevlmrl,
     title = {DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving},
     author = {Huang, Zilin and Sheng, Zihao and Wan, Zhengyang and Qu, Yansong and You, Junwei and Jiang, Sicong and Chen✉, Sikai},
     journal = {arXiv preprint arXiv:2603.18315},
     year = {2026},
     priority = {0},
   }

2025

1. JICV

   

   Sky-Drive: A Distributed Multi-Agent Simulation Platform for Socially-Aware and Human-AI Collaborative Future Transportation

   Zilin Huang^*, Zihao Sheng^*, Zhengyang Wan^*, and 10 more authors

   Journal of Intelligent and Connected Vehicles · 2025

   Abs Paper

   Scholar 13 GitHub 7 ★

   Recent advances in autonomous system simulation platforms have significantly enhanced the safe and scalable testing of driving policies. However, existing simulators do not yet fully meet the needs of future transportation research, particularly in modeling socially-aware driving agents and enabling effective human-AI collaboration. This paper introduces Sky-Drive, a novel distributed multi-agent simulation platform that addresses these limitations through four key innovations: (a) a distributed architecture for synchronized simulation across multiple terminals; (b) a multi-modal human-in-the-loop framework integrating diverse sensors to collect rich behavioral data; (c) a human-AI collaboration mechanism supporting continuous and adaptive knowledge exchange; and (d) a digital twin (DT) framework for constructing high-fidelity virtual replicas of real-world transportation environments. Sky-Drive supports diverse applications such as autonomous vehicle (AV)-vulnerable road user (VRU) interaction modeling, human-in-the-loop training, socially-aware reinforcement learning, personalized driving policy, and customized scenario generation. Future extensions will incorporate foundation models for context-aware decision support and hardware-in-the-loop (HIL) testing for real-world validation. By bridging scenario generation, data collection, algorithm training, and hardware integration, Sky-Drive has the potential to become a foundational platform for the next generation of socially-aware and human-centered autonomous transportation researc

   @article{huang2025sky,
     title = {Sky-Drive: A Distributed Multi-Agent Simulation Platform for Socially-Aware and Human-AI Collaborative Future Transportation},
     author = {Huang, Zilin and Sheng, Zihao and Wan, Zhengyang and Qu, Yansong and Luo, Yuhao and Wang, Boyue and Li, Pei and Chen, Yen-Jung and Chen, Jiancong and Long, Keke and Meng, Jiayi and Leng, Yue and Chen✉, Sikai},
     journal = {Journal of Intelligent and Connected Vehicles},
     year = {2025},
     priority = {1},
     url = {https://www.sciopen.com/article/10.26599/JICV.2026.9210070},
   }

2024

1. J. Automob. Eng.

   

   Study of the Tire Wear of Virtual Track Train

   Hechao Zhou, Zhengyang Wan, Mingsu Mei, and 2 more authors

   Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering · 2024

   Abs Paper

   Scholar 0

   The Virtual Track Train (VTT) represents a novel urban transportation solution that combines elements of both road and rail vehicles. Currently, VTT has gained significant adoption in several cities in China including Yibin, Zhuzhou, Harbin, and Yancheng. Despite its growing usage, the unique structure and complex tire-ground interaction of VTT pose significant challenges in terms of tire wear, which has become a pressing issue. This study develops and validates a multi-body dynamics model of VTT along with a finite element model of its tire. The tire-ground contact behavior and boundary conditions are studied from various perspectives to calculate tire wear. The simulation results demonstrate that the internal pressure of the tires, the load of VTT, the pavement conditions, the inclination angle, and slippage of tires all have critical effects on VTT’s tire wear. These findings suggest that the operating conditions of VTT must be carefully managed to reduce tire wear and ensure safe operation. Furthermore, the research results of tire wear mechanism of VTT offers valuable insights for the optimization and maintenance of VTT.

   @article{hechao2024tirewear,
     priority = {2},
     title = {Study of the Tire Wear of Virtual Track Train},
     author = {Zhou, Hechao and Wan, Zhengyang and Mei, Mingsu and Zhang✉, Jimin and Hecht, Markus},
     journal = {Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering},
     volume = {239},
     issue = {10-11},
     pages = {4608--4621},
     year = {2024},
     publisher = {Sage},
     url = {https://journals.sagepub.com/doi/abs/10.1177/09544070241280744},
   }

2. CSCD

   

   Development of Driving Simulation Platform for Virtual Track Train

   Zhengyang Wan, Hechao Zhou, and Jimin Zhang✉

   Urban Rapid Rail Transit · 2024

   Abs Paper

   The Virtual track train (VTT) has been put into commercial operation as a new type of urban rail transport in China in recent years. To further understand VTT’s operation performance, it is essential to develop a specific driving simulation platform to simulate VTT’s unique hybrid driving mode. The main contribution of this paper are as follows: First, a dynamics model of VTT was established using SIMAPCK. Then, the control module for simulating the hybrid driving mode of VTT was developed based on the model predictive control and Logitech driving simulation hardware. Furthermore, a driving scene visualization interface was established using Unreal Engine and Blender to simulate the real driving environment. These contents constitute the driving simulation platform for VTT. The simulation results show that the developed platform can deliver accurate and comprehensive operation performance of VTT under different driving modes, which has strong engineering significance.

   @article{zhengyang2024vtt,
     priority = {2},
     title = {Development of Driving Simulation Platform for Virtual Track Train},
     author = {Wan, Zhengyang and Zhou, Hechao and Zhang✉, Jimin},
     journal = {Urban Rapid Rail Transit},
     volume = {0},
     pages = {549--560},
     year = {2024},
     dimensions = {true}
   }