Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal-Free Dynamics Fix

• URL: http://arxiv.org/abs/2607.06925v1
• Date: Wed, 08 Jul 2026 02:38:43 GMT
• Title: Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal-Free Dynamics Fix
• Abstract summary: We show that a goal-conditioned predictor reaches a striking $0.90$ relation-readout accuracy, yet this is emphinstruction transcription, not perception.<n>Withholding the goal collapses it to chance ($0.90!to!0.27$, three seeds) and a counterfactual instruction makes the predicted anchors follow the emphfalse instruction $94.5%$ of the time.<n>The diagnosis prescribes the fix: keep the goal out of the dynamics (it belongs to the planner's cost) and supervise the emphread path
• Score: 46.122203287541005
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compact world models that condition on a language goal promise to ground relations such as ``put the red block left of the blue block'' using a sparse set of explicit \emph{reference anchors}. We ask when such references actually ground a relation, and identify a trap: a goal-conditioned predictor reaches a striking $0.90$ relation-readout accuracy, yet this is \emph{instruction transcription}, not perception. Withholding the goal collapses it to chance ($0.90\!\to\!0.27$, three seeds) and a counterfactual instruction makes the predicted anchors follow the \emph{false} instruction $94.5\%$ of the time (true scene $2.3\%$; $N{=}256$). Tested across three settings and a within-task ablation, our central claim characterizes the confound: \textbf{instruction leakage occurs when the scored quantity is transcribable from the instruction (when the instruction names the answer) and is essentially independent of how predictive the non-instruction inputs are.} Our tabletop and the external BabyAI benchmark leak, whereas a Language-Table forward-dynamics world model whose instruction names \emph{referents} does not, until the instruction is augmented to name the direction; and degrading the action never increases leakage, the opposite of what predictor-competition predicts. The diagnosis prescribes the fix: keep the goal out of the dynamics (it belongs to the planner's cost) and supervise the \emph{read} path, recovering genuine, instruction-independent grounding ($0.88$, identical with and without the goal). The detection protocol and remedy apply to any goal-conditioned world model whose instruction names the scored quantity.

Related papers

• Behavior--Realization Separation for Constrained Physical Human--Robot Interaction [0.0]
  This paper implements a receding-horizon quadratic program realizing memoryless affine behaviors.<n>On a torque-controlled 7-DOF Franka FR3 in MuJoCo, the runtime freezes task-space dynamics per solve and enforces torque feasibility across its horizon.
  arXiv  Detail & Related papers  (2026-09-01T03:46:05Z)
• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator [4.010371060637209]
  Spaghetti Architect maps a clean, language-agnostic intermediate representation to deliberately redundant, fully-flattened programs.<n>Every program is compiled, run, and checked against a reference oracle, so each instance is correct by construction.<n>Report baselines on a four-model open ladder: exact match rises with scale, and the intrinsic knob collapses arithmetic-aggregation accuracy of even the strongest model to zero.
  arXiv  Detail & Related papers  (2026-07-21T02:23:22Z)
• Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments [53.85970147743299]
  Expert demonstrations are widely assumed to be the gold standard for robot imitation learning.<n>Yet for fine-grained manipulation such as insertion, stacking, and alignment, we uncover a counterintuitive failure mode: fluent demonstrations can be poor teachers.<n>We introduce STAIR, a compact dynamic feature that bridges the vision-language model and the action expert, distilling the short-horizon motion already recorded in each trajectory into dense, motion-aware supervision.
  arXiv  Detail & Related papers  (2026-06-14T04:15:29Z)
• A Mathematical Theory of Value: a synthesis on goal-directed agency under resource constraints [6.057587531186626]
  We show that value, the quantity goal-directed agents create, destroy, and exchange, is a lawful structural quantity in the same category as information.<n>For populations, value is frame-relative while price is frame-independent; a fleet that pools its resource and fuses its perception inherits the ceiling.
  arXiv  Detail & Related papers  (2026-06-10T16:11:04Z)
• Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override [2.892857112058156]
  Glossaries, technical specifications, and system prompts routinely ask language models to use familiar words in unfamiliar ways.<n>When this works, the lexical prior persists through override rather than being replaced.<n>We test this with a Stroop-style paradigm: a remapping rule pitted against the query word's lexical-prioror ("hospital"), with matched neutral controls.
  arXiv  Detail & Related papers  (2026-05-25T05:53:31Z)
• Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL [76.45061154544568]
  Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels.<n>Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood.<n>We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted.
  arXiv  Detail & Related papers  (2026-05-21T09:19:23Z)
• Subtle Injection for Ground-truth Inference of LLM Training Data [0.2538209532048867]
  SIGIL is a framework that embeds imperceptible emphcanary sequences into protected text and code.<n> SIGIL defines five canary strategies -- lexical-rare, lexical-phrase, syntactic, semantic, and code-pattern.
  arXiv  Detail & Related papers  (2026-05-18T14:48:44Z)
• Representation Without Reward: A JEPA Audit for LLM Fine-Tuning [1.2691047660244335]
  Joint-embedding predictive architectures (JEPAs) propose that a model should learn more useful abstractions when trained to predict latent representations rather than observed outputs.<n>For autoregressive language-model fine-tuning the principle entails a stricter requirement: the induced hidden-state geometry must reach the language-model head emphand improve the decoded task metric.<n>We test that requirement under a fixed Llama-3.2-1B-Instruct LoRA harness on natural-language-to-regex generation.
  arXiv  Detail & Related papers  (2026-05-14T20:27:32Z)
• Language models recognize dropout and Gaussian noise applied to their activations [74.06294367754748]
  We verbalize either (a) emphmask activations, simulating emphdropout, or (b) add emphGaussian noise to them.<n>We test models from the Llama, Olmo, and Qwen families, with sizes between 8B and 32B.
  arXiv  Detail & Related papers  (2026-04-19T14:30:13Z)
• Towards Variable-Length Textual Adversarial Attacks [68.27995111870712]
  It is non-trivial to conduct textual adversarial attacks on natural language processing tasks due to the discreteness of data. In this paper, we propose variable-length textual adversarial attacks(VL-Attack) Our method can achieve $33.18$ BLEU score on IWSLT14 German-English translation, achieving an improvement of $1.47$ over the baseline model.
  arXiv  Detail & Related papers  (2021-04-16T14:37:27Z)
• Toward Adversarial Robustness via Semi-supervised Robust Training [93.36310070269643]
  Adrial examples have been shown to be the severe threat to deep neural networks (DNNs) We propose a novel defense method, the robust training (RT), by jointly minimizing two separated risks ($R_stand$ and $R_rob$)
  arXiv  Detail & Related papers  (2020-03-16T02:14:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.