Learning to Drive via Real-World Simulation at Scale

Haochen Tian^1,2,3^†    Tianyu Li²    Haochen Liu³    Jiazhi Yang²    Yihang Qiu²^†
Guang Li³    Junli Wang^1,3    Yinfeng Gao^1,3    Zhang Zhang¹    Liang Wang¹    Hangjun Ye³
Tieniu Tan¹    Long Chen³    Hongyang Li²
¹MAIS, Institute of Automation, Chinese Academy of Sciences
²OpenDriveLab at The University of HongKong      ³Xiaomi EV
^†Work done while interning at Xiaomi Embodied Intelligence Team

Paper arXiv Code (Dec. 2025) Dataset (Dec. 2025)

Scaling Up End-to-End Planners by Simulation.
(a) We construct large-scale simulation data by perturbing ego trajectories, generating corresponding pseudo-expert demonstrations, and rendering multi-view observations in reactive environments. Combined with real-world data, this enables broad coverage of out-of-distribution states and supports sim–real co-training for any end-to-end planner.
(b) Across three representative planner families, including regression, diffusion, and vocabulary scoring, sim-real co-training consistently produces synergistic improvements in robustness and generalization, demonstrating clear and predictable simulation scaling trends.

Abstract

Achieving fully autonomous driving systems requires learning rational decisions in a wide span of scenarios, including safety-critical and out-of-distribution ones. However, such cases are underrepresented in real-world corpus collected by human experts. To complement for the lack of data diversity, we introduce a novel and scalable simulation framework capable of synthesizing massive unseen states upon existing driving logs. Our pipeline utilizes advanced neural rendering with a reactive environment to generate high-fidelity multi-view observations controlled by the perturbed ego trajectory. Furthermore, we develop a pseudo-expert trajectory generation mechanism for these newly simulated states to provide action supervision. Upon the synthesized data, we find that a simple co-training strategy on both real-world and simulated samples can lead to significant improvements in both robustness and generalization for various planning methods on challenging real-world benchmarks, up to +6.8 EPDMS on navhard and +2.9 on navtest. More importantly, such policy improvement scales smoothly by increasing simulation data only, even without extra real-world data streaming in. We further reveal several crucial findings of such a sim-real learning system, which we term SimScale, including the design of pseudo-experts and the scaling properties for different policy architectures.

Simulation Data Pipeline

Pseudo-Expert Scene Simulation. (a) Trajectory perturbation on T to T + H, (b) reactive environment rollout, and pseudo-expert trajectory generation from T + H to T + 2H under recovery-based and planner-based strategies.

Leaderboard Results

NAVSIMv2 navhard

*: pseudo-expert supervision; †: reward scoring; S.: per-stage EPDM score.

NAVSIMv2 navtest

*: pseudo-expert supervision; †: reward scoring.

Scaling Property Analysis

Scaling Dynamics with Different Planners and Pseudo-Expert Trajectories. Visualization how simulation data scale and supervision signals influence the driving performance of various planners, where the infection point indicates learning plateau

Qaulitative Results

Pseudo-Expert Scene Simulation

Sim. 1

Sim. 2

Sim. 3

Simulated OOD Scenes

More Results

Ecosystem

The code and simulation data would be open-sourced before 2026!

Please stay tuned for more work from OpenDriveLab: R2SE, ReSim, NEXUS, MTGS, Centaur .

BibTeX

If you find the project helpful for your research, please consider citing our paper:

@article{tian2025SimScale,
        title={SimScale: Learning to Drive via Real-World Simulation at Scale},
        author={xxxx},
        journal={arXiv preprint arXiv:xxx},
        year={2025}
      }