Logo Diffusion-Planner

Diffusion-Based Planning for Autonomous Driving with Flexible Guidance

Yinan Zheng*†,1, Ruiming Liang*2, Kexin Zheng*3, Jinliang Zheng1, Liyuan Mao4, Jianxiong Li1,
Weihao Gu5, Rui Ai5, Shengbo Eben Li1, Xianyuan Zhan✉,1,,6 Jingjing Liu✉,1,

1Tsinghua University 2Institute of Automation, Chinese Academy of Sciences
3The Chinese University of Hong Kong 4Shanghai Jiao Tong University
5HAOMO.AI 6Shanghai Artificial Intelligence Laboratory

*Equal contribution, ✉Corresponding author
†Project Lead: zhengyn23@mails.tsinghua.edu.cn

Accepted by ICLR 2025

Arxiv

🔗

 Dataset (Comming Soon) Code Model

100s Closed Loop Perfromance of Diffusion Planner without Refinement

Future planning of the ego vehicle; Predictions for neighboring vehicles; Ground truth ego trajectory; Driving history of the ego vehicle.

Abstract

Achieving human-like driving behaviors in complex open-world environments is a critical challenge in autonomous driving. Contemporary learning-based planning approaches such as imitation learning methods often struggle to balance competing objectives and lack of safety assurance, due to limited adaptability and inadequacy in learning complex multi-modal behaviors commonly exhibited in human planning, not to mention their strong reliance on the fallback strategy with predefined rules. We propose a novel transformer-based Diffusion Planner for closed-loop planning, which can effectively model multi-modal driving behavior and ensure trajectory quality without any rule-based refinement. Our model supports joint modeling of both prediction and planning tasks under the same architecture, enabling cooperative behaviors between vehicles. Moreover, by learning the gradient of the trajectory score function and employing a flexible classifier guidance mechanism, Diffusion Planner effectively achieves safe and adaptable planning behaviors. Evaluations on the large-scale real-world autonomous planning benchmark nuPlan and our newly collected 200-hour delivery-vehicle driving dataset demonstrate that Diffusion Planner achieves SOTA closed-loop performance with robust transferability in diverse driving styles.

Methods

Architecture

Diffusion Planner leverages the expressive and flexible diffusion model to enhance autonomous planning:

  • DiT-based architecture focusing on the fusion of noised future vehicle trajectories and conditional information;
  • Joint modeling of key participants' statuses, unifying motion prediction and closed-loop planning as future trajectory generation;
  • Fast inference during diffusion sampling, achieving around 20Hz for real-time performance

algebraic reasoning

Figure 1: Diffusion Planner Architectuer

Planning Behavior Alignment via Classifier Guidance

Enforcing versatile and controllable driving behavior is crucial for real-world autonomous driving. For example, vehicles must ensure safety and comfort while adjusting speeds to align with user preferences. Thanks to its close relationship to Energy-Based Models, diffusion model can conveniently inject such preferences via classifier guidance. We briefly describe some applicable energy functions that can be used to customize the planning behavior of the model.

  • Target speed maintenance: The speed difference is used as the energy, calculated by comparing the planned average speed with the set target speed.
  • Comfort: The energy function is calculated by measuring the amount by which the vehicle's state exceeds the predefined limits.
  • Collision avoidance: The signed distance between the ego vehicle and neighboring vehicles is computed at each timestamp.
  • Staying within drivable area: The distance the ego vehicle deviates outside the lane at each time step is calculated.
Additionally, this training-free approach supports flexible combinations during inference time, providing a solution for controllable trajectory generation in complex scenarios. For example, as shown in Figure 2, under collision guidance alone, the ego vehicle veers off the road to avoid a rear-approaching vehicle. However, when drivable guidance is added, the vehicle stays on the road while maintaining safety.

Illustration

Figure 2: Starting from the same position, the trajectories driven under different guidance settings.

Case studies for collision guidance: Starting from the same position, we visualized the closed-loop test results. The dashed line with the blurred yellow car represents the results without guidance, while the solid line with the solid car represents the results with guidance.

Case studies for drivable guidance: Starting from the same position, we visualized the closed-loop test results. The dashed line with the blurred yellow car represents the results without guidance, while the solid line with the solid car represents the results with guidance.

algebraic reasoning

Case studies for speed and comfort guidance: For target speed guidance, the speed changes before and after guidance are plotted. For comfort guidance, the longitudinal jerk changes are compared before and after applying comfort guidance on top of collision avoidance guidance.

Experiment Results

SOTA Closed-loop Performance on nuPlan Dataset

algebraic reasoning

Table 1: Closed-loop planning results on nuPlan dataset. Blue: The highest scores of baselines in various types.
*: Using pre-searched reference lines as model input provides prior knowledge, reducing the difficulty of planning compared to standard learning-based methods. NR: non-reactive mode. R: reactive mode.

Qualitative Results

algebraic reasoning

Figure 4: Future trajectory generation visualization. A frame from a challenging narrow road turning scenario in the closed-loop test.

Datasets

Delicery-vehicle Dataset from Haomo.AI

algebraic reasoning

Figure 5: Scenario count by type in the delivery-vehicle driving dataset, with representative visualizations.

We collected approximately 200 hours of real-world data using an autonomous logistics delivery vehicle .
The task of the delivery vehicle is similar to that of a robotaxi in nuPlan, as it autonomously navigates a designated route.
During operation, the vehicle must comply with traffic regulations, ensure safety, and complete the delivery as efficiently as possible.

Compared to the vehicles in the nuPlan dataset, the delivery vehicle

  • is smaller, with lower speeds operation;
  • is able to travel on both main roads and bike lanes;
  • could be more frequently interacts with pedestrians and cyclists, while facing a different driving rule

Closed-loop Performance on Delivery-vehicle Dataset

algebraic reasoning

Table 2: Closed-loop planning results on delivery-vehicle driving dataset.

Dataset is comming soon...

BibTeX


        @inproceedings{
        zheng2025diffusion,
        title={Diffusion-Based Planning for Autonomous Driving with Flexible Guidance},
        author={Yinan Zheng and Ruiming Liang and Kexin Zheng and Jinliang Zheng and Liyuan Mao and Jianxiong Li and Weihao Gu and Rui Ai and Shengbo Eben Li and Xianyuan Zhan and Jingjing Liu},
        booktitle={The Thirteenth International Conference on Learning Representations},
        year={2025},
        url={https://openreview.net/forum?id=wM2sfVgMDH}
        }