SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation

• URL: http://arxiv.org/abs/2605.24354v1
• Date: Sat, 23 May 2026 02:30:33 GMT
• Title: SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation
• Abstract summary: We present SparseWorld, a lightweight world model that focuses on predicting only the critical layout of the scene.<n>SparseWorld significantly reduces collision risk and achieves state-of-the-art performance on the open-loop planning metrics of the nuScenes dataset.
• Score: 13.699319434868483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, world models have made significant progress in enhancing end-to-end driving systems through both future situation forecasting and improved scene understanding. However, existing driving world models are typically built upon dense scene representations, causing high computational costs and redundant information. In this paper, we present SparseWorld, a lightweight world model that focuses on predicting only the critical layout of the scene, enabling efficient future forecasting for end-to-end driving systems. SparseWorld first performs autoregressive rollout to forecast future map elements and surrounding agents, enabling the model to learn how driving scenarios evolve over time. It then leverages these predicted futures to refine downstream motion prediction and trajectory planning. Specifically, we propose a Sparse Dreamer that anticipates future instances in the latent space through joint temporal and spatial attention. By interacting with predicted future instances, the motion planner captures more accurate motion patterns and generates more informed and safety-aware trajectories. Extensive experiments demonstrate that SparseWorld significantly reduces collision risk and achieves state-of-the-art performance on the open-loop planning metrics of the nuScenes dataset with a collision rate of 0.05\%. Moreover, it substantially outperforms the baseline method in closed-loop planning metrics on the Bench2Drive benchmark. Supplementary material is available at the project page: https://wryzju.github.io/SparseWorld/.

Related papers

• Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving [52.04950569530877]
  We present Uni-World VLA, a unified vision-language-action model that tightly interleaves future frame prediction and trajectory planning.<n>Our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions.
  arXiv  Detail & Related papers  (2026-03-28T14:39:51Z)
• ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving [40.28153843744977]
  We propose Temporal Residual World Model (TR-World), which focuses on dynamic object modeling.<n>By calculating the temporal residuals of scene representations, the information of dynamic objects can be extracted without relying on detection and tracking.<n>We also propose Future-Guided Trajectory Refinement (FGTR) module, which conducts interaction between prior trajectories and the future BEV features.
  arXiv  Detail & Related papers  (2026-02-11T14:12:26Z)
• FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Model [103.2513470454204]
  FutureX is a pipeline that enhances end-to-end planners to perform complex motion planning via future scene latent reasoning and trajectory refinement.<n>FutureX enhances existing methods by producing more rational motion plans and fewer collisions without compromising efficiency.
  arXiv  Detail & Related papers  (2025-12-12T02:12:49Z)
• Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution [96.25314747309811]
  We introduce SeerDrive, a novel end-to-end framework that jointly models future scene evolution and trajectory planning.<n>Our method first predicts future bird's-eye view (BEV) representations to anticipate the dynamics of the surrounding scene.<n>Two key components enable this: (1) future-aware planning, which injects predicted BEV features into the trajectory planner, and (2) iterative scene modeling and vehicle planning.
  arXiv  Detail & Related papers  (2025-10-13T07:41:47Z)
• World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model [18.56171397212777]
  We present World4Drive, an end-to-end autonomous driving framework that employs vision foundation models to build latent world models.<n>World4Drive achieves state-of-the-art performance without manual perception annotations on open-loop nuScenes and closed-loop NavSim benchmarks.
  arXiv  Detail & Related papers  (2025-07-01T09:36:38Z)
• COME: Adding Scene-Centric Forecasting Control to Occupancy World Model [18.815436110557112]
  World models are critical for autonomous driving to simulate environmental dynamics and generate synthetic data.<n>Existing methods struggle to disentangle ego-vehicle motion (perspective shifts from scene evolvement)<n>We propose to separate environmental changes from ego-motion by leveraging the scene-centric coordinate systems.
  arXiv  Detail & Related papers  (2025-06-16T09:01:09Z)
• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning [63.25627328308978]
  We propose a-temporal prediction network pipeline to generate future occupancy predictions. Compared to current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds. We publicly release our grid occupancy dataset based on nulis to support further research.
  arXiv  Detail & Related papers  (2022-05-06T13:45:32Z)
• Self-Supervised Action-Space Prediction for Automated Driving [0.0]
  We present a novel learned multi-modal trajectory prediction architecture for automated driving. It achieves kinematically feasible predictions by casting the learning problem into the space of accelerations and steering angles. The proposed methods are evaluated on real-world datasets containing urban intersections and roundabouts.
  arXiv  Detail & Related papers  (2021-09-21T08:27:56Z)
• PnPNet: End-to-End Perception and Prediction with Tracking in the Loop [82.97006521937101]
  We tackle the problem of joint perception and motion forecasting in the context of self-driving vehicles. We propose Net, an end-to-end model that takes as input sensor data, and outputs at each time step object tracks and their future level.
  arXiv  Detail & Related papers  (2020-05-29T17:57:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.