TAG: Target-Agnostic Guidance for Stable Object-Centric Inference in Vision-Language-Action Models

• URL: http://arxiv.org/abs/2603.24584v1
• Date: Wed, 25 Mar 2026 17:56:32 GMT
• Title: TAG: Target-Agnostic Guidance for Stable Object-Centric Inference in Vision-Language-Action Models
• Abstract summary: Vision-Language--Action (VLA) policies have shown strong progress in mapping language instructions and visual observations to robotic actions, yet their reliability degrades in cluttered scenes with distractors.<n>We propose TAG (Target-Agnostic Guidance), a simple inference-time guidance mechanism that explicitly reduces distractor- and appearance-induced bias in VLA policies.<n>We evaluate TAG on standard manipulation benchmarks, including LIBERO, LIBERO-Plus, and VLABench, where it consistently improves robustness under clutter and reduces near-miss and wrong-object executions.
• Score: 59.13964209628383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision--Language--Action (VLA) policies have shown strong progress in mapping language instructions and visual observations to robotic actions, yet their reliability degrades in cluttered scenes with distractors. By analyzing failure cases, we find that many errors do not arise from infeasible motions, but from instance-level grounding failures: the policy often produces a plausible grasp trajectory that lands slightly off-target or even on the wrong object instance. To address this issue, we propose TAG (Target-Agnostic Guidance), a simple inference-time guidance mechanism that explicitly reduces distractor- and appearance-induced bias in VLA policies. Inspired by classifier-free guidance (CFG), TAG contrasts policy predictions under the original observation and an object-erased observation, and uses their difference as a residual steering signal that strengthens the influence of object evidence in the decision process. TAG does not require modifying the policy architecture and can be integrated with existing VLA policies with minimal training and inference changes. We evaluate TAG on standard manipulation benchmarks, including LIBERO, LIBERO-Plus, and VLABench, where it consistently improves robustness under clutter and reduces near-miss and wrong-object executions.

Related papers

• VLA-Corrector: Stage-Aware Observable State Understanding for Prompt-Based Closed-Loop Recovery of Vision-Language-Action Policies [12.698552476812855]
  Long-horizon robot manipulation with Vision-Language-Action (VLA) policies remains vulnerable to execution-time deviations.<n>We introduce a stage-aware failure verification and Prompt Recovery framework that enables closed-loop correction of a fixed VLA policy.
  arXiv  Detail & Related papers  (2026-09-06T09:50:30Z)
• Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring [35.34120853605602]
  textbf-and-Seek is a framework that formulates VLA failure detection as a coarsely supervised learning problem.<n>We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies.
  arXiv  Detail & Related papers  (2026-05-29T04:40:12Z)
• STRIDE: Learnable Stepwise Language Feedback for LLM Reasoning [80.78140312980484]
  We propose language-driven stepwise trajectory redirection, termed as STRIDE.<n>We co-train a generator and a generative verifier using only outcome-based rewards, eliminating external annotations.<n> Experiments on diverse reasoning benchmarks show that STRIDE significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-05-13T11:04:31Z)
• LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models [15.955490895662384]
  We introduce LIBERO-Para, a benchmark for fine-grained analysis of linguistic generalization.<n>We observe consistent performance degradation of 22-52 pp under paraphrasing.<n>We propose PRIDE, a metric that quantifies paraphrase difficulty using semantic and syntactic factors.
  arXiv  Detail & Related papers  (2026-03-30T11:27:34Z)
• Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration [24.562540060971273]
  Vision-Language-Action (VLA) models enable robots to perform manipulation tasks directly from natural language instructions.<n>We reveal a critical failure mode in which VLA policies continue executing visually plausible actions even when the language instruction contradicts the scene.<n>We propose Instruction-Guided Attention Recalibration (IGAR), a train-free inference-time mechanism that rebalances attention to restore the influence of language instructions.
  arXiv  Detail & Related papers  (2026-03-06T08:01:36Z)
• Robust Skills, Brittle Grounding: Diagnosing Restricted Generalization in Vision-Language Action Policies via Multi-Object Picking [6.713814849613683]
  Vision-language action (VLA) policies often report strong manipulation benchmark performance with relatively few demonstrations.<n>We present a controlled multi-object picking study that progressively increases object placement variability up to full workspace randomization.<n>We find that for representative VLA policies, including SmolVLA and $_0.5$, execution of the manipulation primitive remains substantially more reliable than instruction-conditioned task success.
  arXiv  Detail & Related papers  (2026-02-27T16:20:04Z)
• MeGU: Machine-Guided Unlearning with Target Feature Disentanglement [73.49657372882082]
  We propose a novel framework that guides unlearning through concept-aware re-alignment.<n>MeGU enables controlled and selective forgetting, effectively mitigating both under-unlearning and over-unlearning.
  arXiv  Detail & Related papers  (2026-02-19T05:20:31Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models [21.133970394496327]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic control.<n>Current test-time scaling (TTS) methods require additional training, verifiers, and multiple forward passes, making them impractical for deployment.<n>We propose a simple inference strategy that jointly modulates visual perception and action based on'self-uncertainty'
  arXiv  Detail & Related papers  (2026-02-04T04:48:16Z)
• From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models [42.553888395227766]
  We propose a lightweight, training-free framework for vision-language-action (VLA) models.<n>VLA-SCT operates as a self-correcting control loop, combining data-driven action refinement with conditional logic for termination.<n>Our method achieves consistent improvements across all datasets in the LIBERO benchmark.
  arXiv  Detail & Related papers  (2026-02-02T08:44:40Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• LLM-Assisted Semantic Guidance for Sparsely Annotated Remote Sensing Object Detection [25.9348571356454]
  LLM-assisted semantic guidance framework tailored for sparsely annotated remote sensing object detection.<n>Dense Pseudo-Label Assignment mechanism adaptively assigns pseudo-labels for both unlabeled and sparsely labeled data.<n> Adaptive Hard-Negative Reweighting Module to stabilize the supervised learning branch.
  arXiv  Detail & Related papers  (2025-09-21T08:05:43Z)
• Do What? Teaching Vision-Language-Action Models to Reject the Impossible [53.40183895299108]
  Vision-Language-Action (VLA) models have demonstrated strong performance on a range of robotic tasks.<n>We propose Instruct-Verify-and-Act (IVA), a framework that detects when an instruction cannot be executed due to a false premise.<n>Our experiments show that IVA improves false premise detection accuracy by 97.56% over baselines.
  arXiv  Detail & Related papers  (2025-08-22T10:54:33Z)
• MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detection [107.15164718585666]
  We investigate the root cause of VLMs' biased prediction under the open vocabulary detection context. Our observations lead to a simple yet effective paradigm, coded MarvelOVD, that generates significantly better training targets. Our method outperforms the other state-of-the-arts by significant margins.
  arXiv  Detail & Related papers  (2024-07-31T09:23:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.