DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale

• URL: http://arxiv.org/abs/2604.00813v2
• Date: Tue, 07 Apr 2026 14:59:28 GMT
• Title: DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale
• Abstract summary: We propose an alternative Vision-Geometry-Action paradigm that advocates dense 3D geometry as the critical cue for autonomous driving.<n>We introduce a streaming Driving Visual Geometry Transformer (DVGT-2), which processes inputs in an online manner and jointly outputs dense geometry and trajectory planning for the current frame.<n>Despite the faster speed, DVGT-2 achieves superior geometry reconstruction performance on various datasets.
• Score: 63.05446464787182
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end autonomous driving has evolved from the conventional paradigm based on sparse perception into vision-language-action (VLA) models, which focus on learning language descriptions as an auxiliary task to facilitate planning. In this paper, we propose an alternative Vision-Geometry-Action (VGA) paradigm that advocates dense 3D geometry as the critical cue for autonomous driving. As vehicles operate in a 3D world, we think dense 3D geometry provides the most comprehensive information for decision-making. However, most existing geometry reconstruction methods (e.g., DVGT) rely on computationally expensive batch processing of multi-frame inputs and cannot be applied to online planning. To address this, we introduce a streaming Driving Visual Geometry Transformer (DVGT-2), which processes inputs in an online manner and jointly outputs dense geometry and trajectory planning for the current frame. We employ temporal causal attention and cache historical features to support on-the-fly inference. To further enhance efficiency, we propose a sliding-window streaming strategy and use historical caches within a certain interval to avoid repetitive computations. Despite the faster speed, DVGT-2 achieves superior geometry reconstruction performance on various datasets. The same trained DVGT-2 can be directly applied to planning across diverse camera configurations without fine-tuning, including closed-loop NAVSIM and open-loop nuScenes benchmarks.

Related papers

• DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving [89.17723223258513]
  Video panoptic segmentation (DVPS) unifies metric depth, semantic segmentation, and instance trajectories.<n>Existing approaches often rely on computationally expensive, multi-stage pipelines or offline tracking, rendering them unsuitable for real-time decision-making.<n>We propose DVPSFormer, a unified online architecture designed for efficient 4D scene understanding.
  arXiv  Detail & Related papers  (2026-07-28T18:16:00Z)
• PixelPilot: Scalable Vision-Language-Action Models for End-to-End Autonomous Driving [19.811606655441775]
  Vision-Language-Action Models (VLAs) show promising generalization in complex autonomous driving scenarios.<n>VLAs typically predict and optimize 3D trajectories from 2D images.<n>We propose PixelPilot, a novel VLA featuring a decoupled planning and lifting paradigm.
  arXiv  Detail & Related papers  (2026-07-06T03:40:48Z)
• LiAuto-GeoX: Efficient Grounded Driving Transformer [54.23823436153608]
  We present textbfLiAuto-GeoX, an efficient grounded driving transformer for deployable, ego-centric 3D scene understanding.<n>We show that textbfLiAuto-GeoX runs at 220 FPS on KITTI while maintaining high-fidelity dense reconstruction, enabling real-time deployment.
  arXiv  Detail & Related papers  (2026-06-04T06:58:44Z)
• VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Driving [26.557803260279258]
  Cross-view 3D geometric modeling capabilities for autonomous driving is self-evident, yet existing Vision-Language Models inherently lack this capability.<n>We propose a novel architecture, VGGDrive, which empowers Vision-language models with cross-view Geometric Grounding for autonomous driving.
  arXiv  Detail & Related papers  (2026-02-24T11:33:44Z)
• Visual Implicit Geometry Transformer for Autonomous Driving [7.795200422563638]
  We introduce the Visual Implicit Geometry Transformer (ViGT), an autonomous driving geometric model.<n>ViGT estimates a continuous 3D occupancy field in a birds-eye-view (BEV) addressing domain-specific requirements.<n>We validate the scalability and generalizability of our approach by training our model on a mixture of five large-scale autonomous driving datasets.
  arXiv  Detail & Related papers  (2026-02-05T11:54:38Z)
• DVGT: Driving Visual Geometry Transformer [63.38483879291505]
  A driving-targeted dense geometry perception model can adapt to different scenarios and camera configurations.<n>We propose a Driving Visual Geometry Transformer (DVGT), which reconstructs a global dense 3D point map from a sequence of unposed multi-view visual inputs.<n>DVGT is free of explicit 3D geometric priors, enabling flexible processing of arbitrary camera configurations.
  arXiv  Detail & Related papers  (2025-12-18T18:59:57Z)
• VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splatting [47.78433964322689]
  VR-Drive is a novel E2E-AD framework that addresses viewpoint generalization by jointly learning 3D scene reconstruction.<n>Our results demonstrate that VR-Drive is a scalable and robust solution for the real-world deployment of end-to-end autonomous driving systems.
  arXiv  Detail & Related papers  (2025-10-27T10:49:39Z)
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction [86.82819259860186]
  We introduce VLM-3R, a unified framework for Vision-Language Models (VLMs) that incorporates 3D Reconstructive instruction tuning.<n>VLM-3R processes monocular video frames by employing a geometry encoder to derive implicit 3D tokens that represent spatial understanding.
  arXiv  Detail & Related papers  (2025-05-26T17:56:30Z)
• DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation [10.296670127024045]
  DriveScape is an end-to-end framework for multi-view, 3D condition-guided video generation. Our Bi-Directional Modulated Transformer (BiMot) ensures precise alignment of 3D structural information. DriveScape excels in video generation performance, achieving state-of-the-art results on the nuScenes dataset with an FID score of 8.34 and an FVD score of 76.39.
  arXiv  Detail & Related papers  (2024-09-09T09:43:17Z)
• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning [132.20119288212376]
  We propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously. To the best of our knowledge, we are the first to systematically investigate each part of an interpretable end-to-end vision-based autonomous driving system.
  arXiv  Detail & Related papers  (2022-07-15T16:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.