RefGuard: Identity-Aware Language-Guided Robot Manipulation via Joint Target-Anchor-Frame Grounding

• URL: http://arxiv.org/abs/2609.06221v1
• Date: Sat, 05 Sep 2026 18:39:50 GMT
• Title: RefGuard: Identity-Aware Language-Guided Robot Manipulation via Joint Target-Anchor-Frame Grounding
• Abstract summary: RefGuard is a framework that delays commitment by maintaining a joint posterior over all three variables.<n>On a real UF850 arm, RefGuard records no identity switch on any ambiguity-stress trial and executes correctly on 90.0% of them.<n>On a 3200-episode procedural suite, it raises correct execution on solvable instructions from 56.6% to 80.5% over the ablation that commits to the anchor and frame before the target.
• Score: 11.774477485982958
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vision-language-action (VLA) models have substantially advanced language-guided robot manipulation, yet reliable execution still hinges on identifying which physical object an instruction refers to. In cluttered scenes containing repeated objects, ambiguous anchors, or frame-dependent spatial terms, a robot can execute a geometrically valid action on a semantically compatible but unintended instance; we call this failure an identity switch. The referent is jointly determined by three coupled latent variables: the target, the anchor, and the reference frame, so committing to any one of them before execution turns residual ambiguity into a silent and irreversible error. We propose RefGuard, an identity-aware grounding framework that delays commitment by maintaining a joint posterior over all three variables. RefGuard builds a frame-conditioned object-centric scene graph from RGB-D observations, separating frame-independent geometry from directional relations, and routes the posterior through a decision policy that executes, clarifies, reobserves, or aborts. On a real UF850 arm, RefGuard records no identity switch on any ambiguity-stress trial and executes correctly on 90.0% of them, whereas fine-tuned VLA and LLM (Large Language Model)-based baselines switch identity in 33-46% of the same trials, while retaining 93.3% success on unambiguous scenes and recovering from post-grounding scene changes in 86.7% of trials. On a 3200-episode procedural suite, it raises correct execution on solvable instructions from 56.6% to 80.5% over the ablation that commits to the anchor and frame before the target, while deferring less often (19.5% vs. 43.4%).

Related papers

• STAGE: Diagnosing Semantic Transfer at Grounded Execution in Embodied Agents [5.276981655456251]
  Embodied language grounding requires more than identifying the referent of an instruction.<n>We study this missing link as a semantic-action gap, where instruction semantics are recoverable but weakly expressed in native continuous actions.<n>We introduce VISA, a lightweight execution-time interface that converts recovered semantics into ALLOW, DEFER, target-consistency, and verified-selection decisions.
  arXiv  Detail & Related papers  (2026-09-11T19:24:06Z)
• SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure [7.121327089434882]
  SRD-GUARD is a parameter-free, black-box defense framework that exposes concealed intent through semantic rewriting and consensus-based risk assessment.<n>We evaluate SRD-GUARD against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored and DeepSeek-V4-Flash.
  arXiv  Detail & Related papers  (2026-09-06T11:26:08Z)
• Hiding Directions, Leaking Structure: Breaking ArrowCloak through Low-Rank Structure [11.442729593360738]
  TEE-shielded inference keeps sensitive state in a trusted execution environment (TEE) while offloading linear algebra to an untrusted accelerator.<n>Wang et al., in Game of Arrows (USENIX Security 2025), showed that five widely adopted lightweight defenses preserve vector directions and introduced ArrowMatch to exploit this leakage.<n>We propose ArrowCloak, which adds a different multiple of one shared mask direction to each vector and bases its weight-recovery hardness argument on Learning with Errors (LWE)<n>We exploit this structure with our proposed attack, an end-to-end, query-free recovery attack.
  arXiv  Detail & Related papers  (2026-08-21T20:30:26Z)
• StateSight: Benchmarking Latent Spatial-State Reconstruction in Vision-Language Models [0.5624504968545648]
  We introduce StateSight, a benchmark for cube-net opposite-face reasoning, occluded cube-tower counting, and 4-neighbor connected-component counting.<n>Each task family contains 300 single-image prompts with deterministic oracle labels and exact-match scoring.<n>OpenAI GPT-5.5, using the API model identifiergpt-5.5, achieved 59.3%, 33.3%, and 28.3% accuracy across the three tasks.
  arXiv  Detail & Related papers  (2026-08-15T20:13:12Z)
• Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment [42.503300854478795]
  Anchor-Align improves vision-language-action policies on robot demonstrations.<n>It distills layer-wise representations from a frozen VLM copy to prevent this drift.<n>It converts each action target into a discrete motion-direction label and jointly trains language and action prediction.
  arXiv  Detail & Related papers  (2026-07-15T04:13:54Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations [1.2763567932588586]
  ForgetEval is a 1000-case templated suite plus a 385-case adversarial layer (132 hand-crafted + 253 LLM-drafted oracle-validated)<n>We observe three placement regimes with partly complementary coverage: deterministic primitives suffice for lexical/temporal categories but fail canonicalization.<n>We expose the trade-off via ForgetEval, a 1000-case templated suite plus a 385-case adversarial layer.
  arXiv  Detail & Related papers  (2026-06-14T16:32:15Z)
• Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment [45.205992501478256]
  We propose a framework that disentangles holistic supervision into field-specific signals and routes them exclusively to their responsible token spans.<n>Our approach bridges the reliability gap by boosting 3D KPA from 0.64 to 0.93, increasing 3D bounding box intersection over union to 0.686, and raising reprojection consistency scores to 0.852.
  arXiv  Detail & Related papers  (2026-04-23T00:01:40Z)
• ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models [60.14219417402433]
  Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.<n>We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three constructs drawn from standard cognitive-science accounts of non-declarative memory.<n>Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring.
  arXiv  Detail & Related papers  (2026-04-09T10:26:32Z)
• IntRec: Intent-based Retrieval with Contrastive Refinement [42.24292951933703]
  IntRec is an interactive object retrieval framework that refines predictions based on user feedback.<n>At its core is an Intent State (IS) that maintains dual memory sets for positive anchors (confirmed cues) and negative constraints (rejected hypotheses)
  arXiv  Detail & Related papers  (2026-02-19T18:50:53Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.