Clutter-Resistant Vision-Language-Action Models through Object-Centric and Geometry Grounding

• URL: http://arxiv.org/abs/2512.22519v1
• Date: Sat, 27 Dec 2025 08:31:25 GMT
• Title: Clutter-Resistant Vision-Language-Action Models through Object-Centric and Geometry Grounding
• Abstract summary: We propose OBEYED-VLA, a framework that disentangles perceptual grounding from action reasoning.<n>OBEYED-VLA augments Vision-Language-Action models with a perception module that grounds inputs into task-conditioned, object-centric, and geometry-aware observations.<n>On a real-world UR10e tabletop setup, OBEYED-VLA substantially improves robustness over strong VLA baselines.
• Score: 14.290592746124835
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent Vision-Language-Action (VLA) models have made impressive progress toward general-purpose robotic manipulation by post-training large Vision-Language Models (VLMs) for action prediction. Yet most VLAs entangle perception and control in a monolithic pipeline optimized purely for action, which can erode language-conditioned grounding. In our real-world tabletop tests, policies over-grasp when the target is absent, are distracted by clutter, and overfit to background appearance. To address these issues, we propose OBEYED-VLA (OBject-centric and gEometrY groundED VLA), a framework that explicitly disentangles perceptual grounding from action reasoning. Instead of operating directly on raw RGB, OBEYED-VLA augments VLAs with a perception module that grounds multi-view inputs into task-conditioned, object-centric, and geometry-aware observations. This module includes a VLM-based object-centric grounding stage that selects task-relevant object regions across camera views, along with a complementary geometric grounding stage that emphasizes the 3D structure of these objects over their appearance. The resulting grounded views are then fed to a pretrained VLA policy, which we fine-tune exclusively on single-object demonstrations collected without environmental clutter or non-target objects. On a real-world UR10e tabletop setup, OBEYED-VLA substantially improves robustness over strong VLA baselines across four challenging regimes and multiple difficulty levels: distractor objects, absent-target rejection, background appearance changes, and cluttered manipulation of unseen objects. Ablation studies confirm that both semantic grounding and geometry-aware grounding are critical to these gains. Overall, the results indicate that making perception an explicit, object-centric component is an effective way to strengthen and generalize VLA-based robotic manipulation.

Related papers

• HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System [48.02804864175127]
  HiVLA is a visual-grounded-centric hierarchical framework that explicitly decouples high-level semantic planning from low-level motor control.<n>Experiments in simulation and the real world demonstrate that HiVLA significantly outperforms state-of-the-art end-to-end baselines.
  arXiv  Detail & Related papers  (2026-04-15T17:50:07Z)
• SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulation [43.77409401429281]
  Vision-Language-Action (VLA) models show promise for robotic control, yet performance in complex household environments remains sub-optimal.<n>Mobile manipulation requires reasoning about global scene layout, fine-grained geometry, and high-dimensional continuous actions.<n>We introduce a framework for learning spatially-grounded VLA models that strengthens perception and representation.
  arXiv  Detail & Related papers  (2026-03-24T03:44:25Z)
• TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation [70.23578202012048]
  Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch.<n>We propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone.<n>To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment.<n>With the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction.
  arXiv  Detail & Related papers  (2026-03-03T13:28:07Z)
• I-Perceive: A Foundation Model for Active Perception with Language Instructions [41.67607728608853]
  We propose I-Perceive, a foundation model for active perception conditioned on natural language instructions.<n>I-Perceive predicts camera views that follow open-ended language instructions, based on image-based scene contexts.<n> Experiments demonstrate that I-Perceive significantly outperforms state-of-the-art VLMs in both prediction accuracy and instruction following of generated camera views.
  arXiv  Detail & Related papers  (2026-02-28T11:38:56Z)
• Universal Pose Pretraining for Generalizable Vision-Language-Action Policies [83.39008378156647]
  Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency.<n>We propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors.<n>Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion alignment.
  arXiv  Detail & Related papers  (2026-02-23T11:00:08Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• Point What You Mean: Visually Grounded Instruction Policy [42.52502990975079]
  Point-VLA is a plug-and-play policy that augments language instructions with explicit visual cues to resolve referential ambiguity.<n>We evaluate Point-VLA on diverse real-world referring tasks and observe consistently stronger performance than text-only instruction VLAs.
  arXiv  Detail & Related papers  (2025-12-22T00:44:19Z)
• The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs [44.71703930770065]
  We present The Perceptual Observatory, a framework that characterizes MLLMs across verticals like face matching and text-in-vision comprehension capabilities.<n>The Perceptual Observatory moves beyond leaderboard accuracy to yield insights into how MLLMs preserve perceptual grounding and relational structure under perturbations.
  arXiv  Detail & Related papers  (2025-12-17T20:22:23Z)
• ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceiver [35.25196177784228]
  We propose ReconVLA, a reconstructive VLA model with an implicit grounding paradigm.<n>Conditioned on the model's visual outputs, a diffusion transformer reconstructs the gaze region of the image.<n>This process prompts the VLA model to learn fine-grained representations and accurately allocate visual attention.
  arXiv  Detail & Related papers  (2025-08-14T04:20:19Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• Physically Grounded Vision-Language Models for Robotic Manipulation [59.143640049407104]
  We propose PhysObjects, an object-centric dataset of 39.6K crowd-sourced and 417K automated physical concept annotations. We show that fine-tuning a vision-language model on PhysObjects improves its understanding of physical object concepts. We incorporate this physically grounded VLM in an interactive framework with a large language model-based robotic planner.
  arXiv  Detail & Related papers  (2023-09-05T20:21:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.