Learning to Drive via Real-World Simulation at Scale

Haochen Tian^1,2,3^†    Tianyu Li²    Haochen Liu³    Jiazhi Yang²    Yihang Qiu²^†
Guang Li³    Junli Wang^1,3    Yinfeng Gao^1,3    Zhang Zhang¹    Liang Wang¹    Hangjun Ye³
Tieniu Tan¹    Long Chen³    Hongyang Li²
¹MAIS, Institute of Automation, Chinese Academy of Sciences
²OpenDriveLab at The University of HongKong        ³Xiaomi EV
^†Work done while interning at Xiaomi Embodied Intelligence Team

Paper arXiv Code (Dec. 2025) Dataset (Dec. 2025)

TL;DR: Scaling Up End-to-End Planners by Simulation.

🏗️ A scalable simulation pipeline that synthesizes diverse and high-fidelity reactive driving scenarios with pseudo-expert.
🚀 An effective sim-real co-training strategy that improves robustness and generalization across end-to-end planners.
🔬 A comprehensive recipe that reveals crucial insights into the underlying scaling properties of sim-real learning systems.

Abstract

Achieving fully autonomous driving systems requires learning rational decisions in a wide span of scenarios, including safety-critical and out-of-distribution ones. However, such cases are underrepresented in real-world corpus collected by human experts. To complement for the lack of data diversity, we introduce a novel and scalable simulation framework capable of synthesizing massive unseen states upon existing driving logs. Our pipeline utilizes advanced neural rendering with a reactive environment to generate high-fidelity multi-view observations controlled by the perturbed ego trajectory. Furthermore, we develop a pseudo-expert trajectory generation mechanism for these newly simulated states to provide action supervision. Upon the synthesized data, we find that a simple co-training strategy on both real-world and simulated samples can lead to significant improvements in both robustness and generalization for various planning methods on challenging real-world benchmarks, up to +6.8 EPDMS on navhard and +2.9 on navtest. More importantly, such policy improvement scales smoothly by increasing simulation data only, even without extra real-world data streaming in. We further reveal several crucial findings of such a sim-real learning system, which we term SimScale, including the design of pseudo-experts and the scaling properties for different policy architectures.

Simulation Data Pipeline

Pseudo-Expert Scene Simulation. (a) Trajectory perturbation on T to T + H, (b) reactive environment rollout, and pseudo-expert trajectory generation from T + H to T + 2H under recovery-based and planner-based strategies.

Leaderboard Results

NAVSIMv2 navhard

*: pseudo-expert supervision; †: reward scoring; S.: per-stage EPDM score.

NAVSIMv2 navtest

*: pseudo-expert supervision; †: reward scoring.

🏆 #1 NAVSIMv2 Official Leaderboard

NAVSIMv2 Official Leaderboard

More Results

The Effect of Multi-Expert Ensemble

S1/2: Per-stage score; recovery/planner: recovery-based/planner-based expert; reward: reward scoring only; *: ensemble

The Effect of Simulated Reward Scoring

S1/2: per-stage score.

The Effect between Non-reactive vs. Reactive Data Simualtion

#Round: sampling rounds; #Sim.: simulation data number.

Scaling Property Analysis

Scaling Simulation with Fixed Real-World Corpus

Scaling Dynamics with Varying Simulation Data aross End-to-End Planners and Pseudo-Experts.

💡 Pseudo-Expert Should Be Exploratory.

💡 Multi-Modality Modeling Sparks Scaling.

💡 Reward is All You Need.

Scaling Simulation with Fixed Sim-Real Ratio

Scaling Dynamics with Varying Real-World Data across End-to-End Planners.

💡 Sustained Simulation Gains Across Real Data Scales.

Qaulitative Results

Pseudo-Expert Scene Simulation

Sim. 1

Sim. 2

Sim. 3

Simulated OOD Scenes

More Results

Ecosystem

The code and simulation data would be open-sourced before 2026!

Please stay tuned for more work from OpenDriveLab: R2SE, ReSim, NEXUS, MTGS, Centaur .

BibTeX

If you find the project helpful for your research, please consider citing our paper:

@article{tian2025simscale,
        title={SimScale: Learning to Drive via Real-World Simulation at Scale},
        author={Haochen Tian and Tianyu Li and Haochen Liu and Jiazhi Yang and Yihang Qiu and Guang Li and Junli Wang and Yinfeng Gao and Zhang Zhang and Liang Wang and Hangjun Ye and Tieniu Tan and Long Chen and Hongyang Li},
        journal={arXiv preprint arXiv:2511.23369},
        year={2025}
      }