STAGE: Diagnosing Semantic Transfer at Grounded Execution in Embodied Agents

• URL: http://arxiv.org/abs/2609.13458v1
• Date: Fri, 11 Sep 2026 19:24:06 GMT
• Title: STAGE: Diagnosing Semantic Transfer at Grounded Execution in Embodied Agents
• Abstract summary: Embodied language grounding requires more than identifying the referent of an instruction.<n>We study this missing link as a semantic-action gap, where instruction semantics are recoverable but weakly expressed in native continuous actions.<n>We introduce VISA, a lightweight execution-time interface that converts recovered semantics into ALLOW, DEFER, target-consistency, and verified-selection decisions.
• Score: 5.276981655456251
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied language grounding requires more than identifying the referent of an instruction: recovered semantics must also control the action an agent exposes. We study this missing link as a semantic-action gap, where instruction semantics are recoverable but weakly expressed in native continuous actions. We introduce SAT-Bench, a fixed-observation counterfactual benchmark that holds the visual scene and agent state fixed while changing only instruction semantics. On LIBERO target-name and pixel-grounded relation swaps, target recovery reaches 100.0% and 95.8%, whereas OpenVLA action sensitivity remains only 6.8% and 7.7%. The gap persists across 1,000 additional compositional and temporal/procedural counterfactuals, with overall action sensitivity of 6.1%. Hidden-state, threshold-free, cross-policy, and rollout diagnostics further support this semantic-action transfer failure. We introduce VISA, a lightweight execution-time interface that converts recovered semantics into ALLOW, DEFER, target-consistency, and verified-selection decisions. VISA reduces invalid-instruction blind execution from 92.7% to 2.8% while preserving 94.0% of normal commands, and verified selection further improves target-consistent action exposure without updating the underlying policy. Overall, embodied language evaluation should measure semantic-action transfer, not semantic parsing alone.

Related papers

• RefGuard: Identity-Aware Language-Guided Robot Manipulation via Joint Target-Anchor-Frame Grounding [11.774477485982958]
  RefGuard is a framework that delays commitment by maintaining a joint posterior over all three variables.<n>On a real UF850 arm, RefGuard records no identity switch on any ambiguity-stress trial and executes correctly on 90.0% of them.<n>On a 3200-episode procedural suite, it raises correct execution on solvable instructions from 56.6% to 80.5% over the ablation that commits to the anchor and frame before the target.
  arXiv  Detail & Related papers  (2026-09-05T18:39:50Z)
• Ontology-Grounded World Models for Failure Diagnosis and Closed-Loop Repair in Physical AI Systems [0.815557531820863]
  Onto-EV-WM is a diagnosis and verification-gated correction interface layered above EV-WM.<n>Onto-EV-WM succeeds on 8,526 tasks, with suite-level success rates of 65.98% for LIBERO-10, 91.39% for LIBERO-Goal, and 91.38% for both LIBERO-Object and LIBERO-Spatial.
  arXiv  Detail & Related papers  (2026-08-14T03:13:57Z)
• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis [31.25049774595995]
  We study whether refusal behavior remains intact (Aligned) or has been refusalablated (Abliterated) within same-lineage models.<n>We evaluate Aligned and Abliterated states on vulnerability detection, CWE attribution, vulnerableline localization, root-cause localization, and executable patch validation.
  arXiv  Detail & Related papers  (2026-07-07T05:04:48Z)
• ACCORD: Action-Conditioned Contextual Grounding for Language Agents [69.73525608707764]
  ACCORD (Action-Conditioned Contextual Grounding) is an agent framework for adaptive grounding.<n>We show that current agents often fail to do so. They act from assumed rather than observed specifics, overlook information they could have gathered, and fail to incorporate evidence that has already been returned.
  arXiv  Detail & Related papers  (2026-06-15T09:05:55Z)
• Selective Agentic Recovery for UAV Autonomy with a Persistent Mission Runtime [1.189955933770711]
  On physical UAVs, remote reasoning is most useful when it is invoked selectively.<n>This paper presents Persistent Mission (PMR), a recovery framework that keeps the mission loop and safety-critical execution local.<n>PMR introduces learned Cognitive Value of Invocation (learned-CVI), a compact admission gate that estimates when remote agentic reasoning is likely to improve near-term mission progress.
  arXiv  Detail & Related papers  (2026-06-12T07:57:47Z)
• ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents [2.585367539248893]
  We investigate whether the transition from standard execution to a clarification-seeking state increases an agent's susceptibility to prompt injection attacks.<n>We introduce ASPI, a benchmark of 728 task-attack scenarios that isolates clarification as a distinct agent state.<n>We find that clarification-seeking consistently and substantially amplifies vulnerability.
  arXiv  Detail & Related papers  (2026-05-17T08:30:45Z)
• Are Benchmark Tests Strong Enough? Mutation-Guided Diagnosis and Augmentation of Regression Suites [49.16055123488827]
  Insufficiently strong test suites can admit plausible yet semantically incorrect patches, inflating reported success rates.<n>We introduce STING, a framework for targeted test augmentation that uses semantically altered program variants as diagnostic stressors.<n>STING produces 1,014 validated tests spanning 211 instances and increases patch-region line and branch coverage by 10.8% and 9.5%.
  arXiv  Detail & Related papers  (2026-04-02T01:13:40Z)
• When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents [50.5814495434565]
  This work makes the first effort to define and study misaligned action detection in computer-use agents (CUAs)<n>We identify three common categories in real-world CUA deployment and construct MisActBench, a benchmark of realistic trajectories with human-annotated, action-level alignment labels.<n>We propose DeAction, a practical and universal guardrail that detects misaligned actions before execution and iteratively corrects them through structured feedback.
  arXiv  Detail & Related papers  (2026-02-09T18:41:15Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• Defending against Indirect Prompt Injection by Instruction Detection [109.30156975159561]
  InstructDetector is a novel detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks.<n>InstructDetector achieves a detection accuracy of 99.60% in the in-domain setting and 96.90% in the out-of-domain setting, and reduces the attack success rate to just 0.03% on the BIPIA benchmark.
  arXiv  Detail & Related papers  (2025-05-08T13:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.