Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models

• URL: http://arxiv.org/abs/2605.24642v1
• Date: Sat, 23 May 2026 16:18:41 GMT
• Title: Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models
• Abstract summary: Recent work explores new opportunities at the intersection of vision-language-action models (VLAs) and geometric foundation models (GFMs) for 3D reconstruction.<n>It remains unclear (i) if modern VLAs already have sufficient geometric understanding to start with, (ii) what is the best architecture to inject geometric understanding into a VLA, and (iii) what is the effect of other design choices that affect geometric VLAs.<n>This paper provides a rigorous experimental analysis to shed light on these questions, for a specific choice of VLA (GR00T-N1.5) and GFM (VGGT)
• Score: 16.739408045405906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work explores new opportunities at the intersection of vision-language-action models (VLAs) and geometric foundation models (GFMs) for 3D reconstruction, such as VGGT. While the resulting geometric VLAs often show improved performance, it remains unclear (i) if modern VLAs already have sufficient geometric understanding to start with, (ii) what is the best architecture to inject geometric understanding into a VLA, and (iii) what is the effect of other design choices that affect geometric VLAs. In this paper we provide a rigorous experimental analysis to shed light on these questions, for a specific choice of VLA (GR00T-N1.5) and GFM (VGGT). Our first contribution is to formalize prior work's intuition that current VLAs lack geometric understanding, by providing a rigorous analysis based on linear probing. The analysis quantifies, for the first time, the "geometric gap" between VLAs and GFMs. Our second contribution is to identify and compare different strategies to bridge GFMs with VLAs. We implement three different architectures, which differ in the way they inject geometry in the VLA, while keeping low-level implementation details as similar as possible, to ensure a fair comparison. Finally, we analyze the impact of non-architectural choices (e.g., training data, number of cameras, reconstruction quality) on the performance of the geometric VLAs.

Related papers

• Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation [76.16065370700488]
  We introduce Lift3D-VLA, a unified VLA framework that equips models with explicit 3D point cloud reasoning and enables temporally coherent action generation.<n>We show that Lift3D-VLA achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench than the best-performing prior VLA methods.
  arXiv  Detail & Related papers  (2026-07-07T17:59:47Z)
• Last-Layer-Centric Feature Recombination: Unleashing 3D Geometric Knowledge in DINOv3 for Monocular Depth Estimation [4.654162664140336]
  Recent vision foundation models (VFMs) have significantly improved accuracy and generalization for dense prediction.<n>In this study, we present a systematic layer-wise analysis of DINOv3, revealing that 3D information is distributed non-uniformly.<n>Motivated by this, we introduce a Last-Layer-Centric Feature Recombination (LFR) module to enhance geometric expressiveness.
  arXiv  Detail & Related papers  (2026-04-29T09:11:36Z)
• Modeling Cross-vision Synergy for Unified Large Vision Model [130.37489011094036]
  PolyV is a unified large vision model that achieves cross-vision synergy at both the architectural and training levels.<n>PolyV consistently outperforms existing models, achieving over 10% average improvement over its backbone.
  arXiv  Detail & Related papers  (2026-03-03T22:44:43Z)
• VLANeXt: Recipes for Building Strong VLA Models [95.4552662536287]
  Vision-Language-Action models (VLAs) emerged, leveraging strong visual and language understanding for policy learning.<n>Many groups have proposed their own VLA models, but inconsistencies in training protocols and evaluation settings make it difficult to identify which design choices truly matter.<n>We will release a unified, easy-to-use framework that serves as a common platform for the community to reproduce our findings.
  arXiv  Detail & Related papers  (2026-02-20T09:26:17Z)
• On Geometric Understanding and Learned Data Priors in VGGT [38.8968170074396]
  The Visual Geometry Grounded Transformer (VGGT) is a 3D foundation model that infers camera geometry and scene structure in a single feed-forward pass.<n>We conduct a systematic analysis of VGGT's internal mechanisms to uncover whether geometric understanding emerges within its representations.
  arXiv  Detail & Related papers  (2025-12-12T12:11:57Z)
• GLaD: Geometric Latent Distillation for Vision-Language-Action Models [106.53332923530245]
  We introduce GLaD, a geometry-aware Vision-Language-Action (VLA) framework that incorporates 3D geometric priors during pretraining through knowledge distillation.<n>GLaD achieves 94.1% average success rate across four LIBERO task suites, outperforming UniVLA (92.5%) which uses identical pretraining data.
  arXiv  Detail & Related papers  (2025-12-10T13:07:27Z)
• Aligning Large Language Models and Geometric Deep Models for Protein Representation [57.59506688299817]
  Latent representation alignment is used to map embeddings from different modalities into a shared space, often aligned with the embedding space of large language models (LLMs)<n>Preliminary protein-focused large language models (MLLMs) have emerged, but they have predominantly relied on approaches lacking a fundamental understanding of optimal alignment practices across representations.<n>In this study, we explore the alignment of multimodal representations between LLMs and Geometric Deep Models (GDMs) in the protein domain.<n>Our work examines alignment factors from both model and protein perspectives, identifying challenges in current alignment methodologies and proposing strategies to improve the alignment process.
  arXiv  Detail & Related papers  (2024-11-08T04:15:08Z)
• GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation [91.15865862160088]
  We introduce a geometric flow network (GFNet) to explore the geometric correspondence between different views in an align-before-fuse manner. Specifically, we devise a novel geometric flow module (GFM) to bidirectionally align and propagate the complementary information across different views.
  arXiv  Detail & Related papers  (2022-07-06T11:48:08Z)
• Few-shot Learning as Cluster-induced Voronoi Diagrams: A Geometric Approach [12.382578792491747]
  Cluster-induced Voronoi Diagram (CIVD) improves the accuracy and robustness of few-shot learning. Our CIVD-based workflow enables us to achieve new state-of-the-art results on mini-ImageNet, CUB, and tiered-ImagenNet datasets.
  arXiv  Detail & Related papers  (2022-02-05T02:52:06Z)
• Self-supervised Geometric Perception [96.89966337518854]
  Self-supervised geometric perception is a framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels. We show that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.
  arXiv  Detail & Related papers  (2021-03-04T15:34:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.