DVGT: Driving Visual Geometry Transformer

• URL: http://arxiv.org/abs/2512.16919v1
• Date: Thu, 18 Dec 2025 18:59:57 GMT
• Title: DVGT: Driving Visual Geometry Transformer
• Authors: Sicheng Zuo, Zixun Xie, Wenzhao Zheng, Shaoqing Xu, Fang Li, Shengyin Jiang, Long Chen, Zhi-Xin Yang, Jiwen Lu,
• Abstract summary: A driving-targeted dense geometry perception model can adapt to different scenarios and camera configurations.<n>We propose a Driving Visual Geometry Transformer (DVGT), which reconstructs a global dense 3D point map from a sequence of unposed multi-view visual inputs.<n>DVGT is free of explicit 3D geometric priors, enabling flexible processing of arbitrary camera configurations.
• Score: 63.38483879291505
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Perceiving and reconstructing 3D scene geometry from visual inputs is crucial for autonomous driving. However, there still lacks a driving-targeted dense geometry perception model that can adapt to different scenarios and camera configurations. To bridge this gap, we propose a Driving Visual Geometry Transformer (DVGT), which reconstructs a global dense 3D point map from a sequence of unposed multi-view visual inputs. We first extract visual features for each image using a DINO backbone, and employ alternating intra-view local attention, cross-view spatial attention, and cross-frame temporal attention to infer geometric relations across images. We then use multiple heads to decode a global point map in the ego coordinate of the first frame and the ego poses for each frame. Unlike conventional methods that rely on precise camera parameters, DVGT is free of explicit 3D geometric priors, enabling flexible processing of arbitrary camera configurations. DVGT directly predicts metric-scaled geometry from image sequences, eliminating the need for post-alignment with external sensors. Trained on a large mixture of driving datasets including nuScenes, OpenScene, Waymo, KITTI, and DDAD, DVGT significantly outperforms existing models on various scenarios. Code is available at https://github.com/wzzheng/DVGT.

Related papers

• Geometry-Aware Rotary Position Embedding for Consistent Video World Model [48.914346802616414]
  ViewRope is a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers.<n>Geometry-Aware Frame-Sparse Attention exploits these geometric cues to selectively attend to relevant historical frames.<n>Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.
  arXiv  Detail & Related papers  (2026-02-08T08:01:16Z)
• Visual Implicit Geometry Transformer for Autonomous Driving [7.795200422563638]
  We introduce the Visual Implicit Geometry Transformer (ViGT), an autonomous driving geometric model.<n>ViGT estimates a continuous 3D occupancy field in a birds-eye-view (BEV) addressing domain-specific requirements.<n>We validate the scalability and generalizability of our approach by training our model on a mixture of five large-scale autonomous driving datasets.
  arXiv  Detail & Related papers  (2026-02-05T11:54:38Z)
• GPA-VGGT:Adapting VGGT to Large Scale Localization by Self-Supervised Learning with Geometry and Physics Aware Loss [15.633839321933385]
  Recent advancements in Visual Geometry Grounded Transformer (VGGT) models have shown great promise in camera pose estimation and 3D reconstruction.<n>These models typically rely on ground truth labels for training, posing challenges when adapting to unlabeled and unseen scenes.<n>We propose a self-supervised framework to train VGGT with unlabeled data, thereby enhancing its localization capability in large-scale environments.
  arXiv  Detail & Related papers  (2026-01-23T16:46:59Z)
• 360Anything: Geometry-Free Lifting of Images and Videos to 360° [51.50120114305155]
  Existing approaches rely on explicit geometric alignment between the perspective and the equirectangular projection space.<n>We propose 360Anything, a geometry-free framework built upon pre-trained diffusion transformers.<n>Our approach achieves state-of-the-art performance on both image and video perspective-to-360 generation.
  arXiv  Detail & Related papers  (2026-01-22T18:45:59Z)
• VGGT: Visual Geometry Grounded Transformer [61.37669770946458]
  VGGT is a feed-forward neural network that directly infers all key 3D attributes of a scene.<n>Network achieves state-of-the-art results in multiple 3D tasks.
  arXiv  Detail & Related papers  (2025-03-14T17:59:47Z)
• MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes [72.02827211293736]
  MagicDrive3D is a novel framework for controllable 3D street scene generation.<n>It supports multi-condition control, including road maps, 3D objects, and text descriptions.<n>It generates diverse, high-quality 3D driving scenes, supports any-view rendering, and enhances downstream tasks like BEV segmentation.
  arXiv  Detail & Related papers  (2024-05-23T12:04:51Z)
• DUSt3R: Geometric 3D Vision Made Easy [8.471330244002564]
  We introduce DUSt3R, a novel paradigm for Dense and Unconstrained Stereo 3D Reconstruction of arbitrary image collections.<n>We show that this formulation smoothly unifies the monocular and binocular reconstruction cases.<n>Our formulation directly provides a 3D model of the scene as well as depth information, but interestingly, we can seamlessly recover from it, pixel matches, relative and absolute camera.
  arXiv  Detail & Related papers  (2023-12-21T18:52:14Z)
• MagicDrive: Street View Generation with Diverse 3D Geometry Control [82.69871576797166]
  We introduce MagicDrive, a novel street view generation framework, offering diverse 3D geometry controls. Our design incorporates a cross-view attention module, ensuring consistency across multiple camera views.
  arXiv  Detail & Related papers  (2023-10-04T06:14:06Z)
• Multi-camera Bird's Eye View Perception for Autonomous Driving [17.834495597639805]
  It is essential to produce perception outputs in 3D to enable the spatial reasoning of other agents and structures. The most basic approach to achieving the desired BEV representation from a camera image is IPM, assuming a flat ground surface. More recent approaches use deep neural networks to output directly in BEV space.
  arXiv  Detail & Related papers  (2023-09-16T19:12:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.