CWM: Contrastive World Models for Action Feasibility Learning in Embodied Agent Pipelines

• URL: http://arxiv.org/abs/2602.22452v1
• Date: Wed, 25 Feb 2026 22:27:30 GMT
• Title: CWM: Contrastive World Models for Action Feasibility Learning in Embodied Agent Pipelines
• Abstract summary: A reliable action feasibility scorer is a critical bottleneck in embodied agent pipelines.<n>Existing approaches use supervised fine-tuning (SFT) to train action scorers.<n>We propose Contrastive World Model (CWM), which fine-tunes a large language model (LLM) as an action scorer.
• Score: 2.8681767712152197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A reliable action feasibility scorer is a critical bottleneck in embodied agent pipelines: before any planning or reasoning occurs, the agent must identify which candidate actions are physically executable in the current state. Existing approaches use supervised fine-tuning (SFT) to train action scorers, but SFT treats each candidate independently and does not explicitly teach the model to discriminate between actions that are physically correct and those that are subtly wrong. We propose the Contrastive World Model (CWM), which fine-tunes a large language model (LLM) as an action scorer using an InfoNCE contrastive objective with hard-mined negative examples. The key idea is to push valid actions away from invalid ones in scoring space, with special emphasis on hard negatives: semantically similar but physically incompatible candidates. We evaluate CWM on the ScienceWorld benchmark through two studies. First, an intrinsic affordance evaluation on 605 hard-negative test pairs shows that CWM outperforms SFT by +6.76 percentage points on Precision@1 for minimal-edit negatives -- cases where a single word changes the physical outcome -- and achieves a higher AUC-ROC (0.929 vs. 0.906). Second, a live filter characterisation study measures how well CWM ranks gold-path actions against all valid environment actions during task execution. Under out-of-distribution stress conditions, CWM maintains a significantly better safety margin (-2.39) than SFT (-3.96), indicating that the gold action is ranked closer to the top. These results support the hypothesis that contrastive training induces representations that capture physical feasibility more faithfully than SFT alone.

Related papers

• Overcoming Statistical Bias in Action-Controllable World Models [18.890337378174678]
  Action-conditioned world models aim to predict how visual environments evolve under an agent's actions.<n>Future frames are often highly predictable from visual inertia and recurring motion patterns alone.<n>We introduce CoCo, a Counterfactual Consistency framework to enhance action controllability.
  arXiv  Detail & Related papers  (2026-08-05T10:10:36Z)
• Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning [0.0]
  Intervention bias is a previously unquantified failure mode of large-language-model educational advisory agents.<n>We show that supervised policy learning eliminates this bias.<n>We also show an Evaluation Gap: LLM-as-judge scoring is blind to intervention bias, rewarding fluent over-prescription rather than decision quality.
  arXiv  Detail & Related papers  (2026-06-28T08:58:17Z)
• Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors [15.880694616254297]
  We introduce a benchmark for measuring model propensity for instrumental convergence (IC) behaviour in terminal-based agents.<n>Our benchmark is realistic and low-stakes which serves to reduce evaluation-awareness and roleplay confounds.<n>Our findings indicate that realistic, low-nudge environments elicit IC behaviour rarely but systematically in most tested models.
  arXiv  Detail & Related papers  (2026-05-07T16:12:36Z)
• When Does Margin Clamping Affect Training Variance? Dataset-Dependent Effects in Contrastive Forward-Forward Learning [0.0]
  Contrastive Forward-Forward (CFF) learning trains Vision Transformers layer by layer against supervised contrastive objectives.<n>We focus on one implementation detail: the positive-pair margin in the contrastive loss is applied through saturating similarity clamping.<n>We prove that an alternative formulation, subtracting the margin after the log-probability, is gradient-neutral under the mean-over-positives reduction.
  arXiv  Detail & Related papers  (2026-03-01T07:00:38Z)
• When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents [50.5814495434565]
  This work makes the first effort to define and study misaligned action detection in computer-use agents (CUAs)<n>We identify three common categories in real-world CUA deployment and construct MisActBench, a benchmark of realistic trajectories with human-annotated, action-level alignment labels.<n>We propose DeAction, a practical and universal guardrail that detects misaligned actions before execution and iteratively corrects them through structured feedback.
  arXiv  Detail & Related papers  (2026-02-09T18:41:15Z)
• Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Driving [3.8612647047433217]
  Group Relative Behavior Optimization improves safety performance by over 40% while preserving behavioral realism.<n>Warm-K is a warm-started Top-K sampling strategy that balances consistency and diversity in motion selection.
  arXiv  Detail & Related papers  (2025-12-15T12:18:50Z)
• Compositional Bias Control in Large Language Models: Preference Learning Fails, Supervision Succeeds [0.0]
  Large Language Models (LLMs) still produce gender-stereotyped language even in occupation-neutral contexts.<n>We compare six control techniques for bias mitigation: prompt-only, generate-and-filter, DFA-based Ctrl-G decoding, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Iterative Nullspace Projection (INLP)<n>SFT achieves 99.87 +- 0.15% compliance and high lexical diversity, while DPO, despite similar training stability, fails at 4.53 +- 0.82%.
  arXiv  Detail & Related papers  (2025-10-24T23:52:37Z)
• Supervised Fine-Tuning or Contrastive Learning? Towards Better Multimodal LLM Reranking [56.46309219272326]
  For large language models (LLMs), classification via supervised fine-tuning (SFT) predicts ''yes'' (resp. ''no'') token for relevant (resp. irrelevant) pairs.<n>This divergence raises a central question: which objective is intrinsically better suited to LLM-based reranking, and what mechanism underlies the difference?<n>We conduct a comprehensive comparison and analysis between CL and SFT for reranking, taking the universal multimodal retrieval (UMR) as the experimental playground.
  arXiv  Detail & Related papers  (2025-10-16T16:02:27Z)
• From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs [58.02809208460186]
  We revisit this paradox using high-quality traces from DeepSeek-R1 as demonstrations.<n>We find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal.<n>We introduce Insight-to-solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights.
  arXiv  Detail & Related papers  (2025-09-27T08:59:31Z)
• The Thinking Therapist: Training Large Language Models to Deliver Acceptance and Commitment Therapy using Supervised Fine-Tuning and Odds Ratio Policy Optimization [0.0]
  Acceptance and Commitment Therapy (ACT) is a cognitive behavioral therapy with emerging evidence of efficacy in several psychiatric conditions.<n>This study investigates the impact of post-training methodology and explicit reasoning on the ability of a small open-weight large language model (LLM) to deliver ACT.
  arXiv  Detail & Related papers  (2025-09-08T02:30:12Z)
• Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling [51.38330727868982]
  We show how action chunking impacts the divergence between a learner and a demonstrator.<n>We propose Bidirectional Decoding (BID), a test-time inference algorithm that bridges action chunking with closed-loop adaptation.<n>Our method boosts the performance of two state-of-the-art generative policies across seven simulation benchmarks and two real-world tasks.
  arXiv  Detail & Related papers  (2024-08-30T15:39:34Z)
• Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off [107.35833747750446]
  adversarial examples can be crafted by adding imperceptible perturbations to legitimate documents. This vulnerability raises significant concerns about their reliability and hinders the widespread deployment of NRMs. In this study, we establish theoretical guarantees regarding the effectiveness-robustness trade-off in NRMs.
  arXiv  Detail & Related papers  (2023-12-16T05:38:39Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.