AGWM: Affordance-Grounded World Models for Environments with Compositional Prerequisites

• URL: http://arxiv.org/abs/2605.06841v1
• Date: Thu, 07 May 2026 18:46:44 GMT
• Title: AGWM: Affordance-Grounded World Models for Environments with Compositional Prerequisites
• Abstract summary: We propose AGWM (Affordance-Grounded World Model), which learns an abstract affordance structure represented as a DAG of prerequisite dependencies to explicitly track the dynamic executability of actions.<n> Experiments on game-based simulated environments demonstrate the effectiveness of our method by achieving lower multi-step prediction error, better generalization to novel configurations, and improved interpretability.
• Score: 18.635313325437664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In model-based learning, the agent learns behaviors by simulating trajectories based on world model predictions. Standard world models typically learn a stationary transition function that maps states and actions to next states, when an action and an outcome frequently co-occur in training data, the model tends to internalize this correlation as a general causal rule while ignoring action preconditions. In interactive environments, however, agent actions can reshape the future affordance space. At each timestep, an action may becomes executable only after its prerequisites are met, or non-executable when they are destroyed. We term such events structure-changing events (SC events). As a result, a conventional world model often fails to determine whether a given action is executable in the current state, especially in multi-step predictions. Each imagined step is conditioned on an incorrect affordance state, and therefore the prediction error compounds over the rollout horizon. In this paper, we propose AGWM (Affordance-Grounded World Model), which learns an abstract affordance structure represented as a DAG of prerequisite dependencies to explicitly track the dynamic executability of actions. Experiments on game-based simulated environments demonstrate the effectiveness of our method by achieving lower multi-step prediction error, better generalization to novel configurations, and improved interpretability.

Related papers

• CAER: Causal Action Effect Reweighting for World Model Training [31.737118584204342]
  We introduce Causal Action Effect Reweighting (CAER), a training paradigm that redistributes supervision toward the tokens whose predicted future is causally affected by the action.<n>CAER contrasts the model's own predictions with and without action conditioning to localize these tokens online, then normalizes the resulting effect map into a weight that preserves the total coefficient mass and changes only where it is spent.<n> Experiments across heterogeneous action-conditioned world-model tasks show that CAER converges to better solutions than uniform MSE training, with consistent improvements in the physical consistency, controllability, and visual quality of generated videos
  arXiv  Detail & Related papers  (2026-08-31T14:49:56Z)
• On the Capability Separation Between World-Model Policy Learning and Imitated World-Action Models [3.79187263097166]
  We compare a direct behavior-cloning policy, an imitation-trained world-action policy, and a policy optimized with an action-conditioned world model.<n>Future prediction changes the learning factorization but not the unrestricted external policy class or ideal imitation target.
  arXiv  Detail & Related papers  (2026-08-23T03:25:31Z)
• From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence [54.62791372989525]
  Action models, vision-language--action policies, and world models have advanced this goal.<n>World Action Models (WAMs) are particularly promising because they connect candidate interventions with predicted consequences.<n>We review the evolution toward WAMs and organize these limitations into three coupled gaps: model roles and representations, objectives and standardization, and system composition.
  arXiv  Detail & Related papers  (2026-07-13T15:22:56Z)
• Trimming the Long-Tail of Visual World Modeling Evaluation [67.17191772079316]
  We introduce Tailor-Bench, a benchmark that challenges world models to simulate irregular physical interactions.<n>Regular scenarios reflect common tool-task pairs, Unconventional scenarios replace conventional tools with attribute-compatible substitutes, and Impossible scenarios introduce attribute-violating tools.<n>Our experimental results reveal a clear long-tail gap in physical world modeling, indicating limited generalization beyond common interactions.
  arXiv  Detail & Related papers  (2026-06-23T07:43:41Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• SCAR: Self-Supervised Continuous Action Representation Learning [36.917304453471864]
  We propose SCAR, a joint inverse-forward dynamics framework for learning unified action representations across embodiments from visual transitions.<n>Built on a pretrained generative backbone, SCAR uses an inverse dynamics model (IDM) to infer latent actions from latent observation pairs and a forward dynamics model (FDM) to predict future dynamics conditioned on them.<n>Experiments on the Procgen and Robotwin dataset show that the learned unified latent action representation serves as a stronger conditioning interface for world modeling than embodiment-specific raw actions.
  arXiv  Detail & Related papers  (2026-05-13T16:23:11Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• Learning Lifted Action Models from Unsupervised Visual Traces [38.479081137300945]
  We propose a deep learning framework that jointly learns state prediction, action prediction, and a lifted action model.<n>We also introduce a mixed-integer linear program (MILP) to prevent prediction collapse and self-reinforcing errors.<n>Experiments across multiple domains show that integrating MILP-based correction helps the model escape local optima and converge toward globally consistent solutions.
  arXiv  Detail & Related papers  (2026-04-21T03:49:04Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Specification-Driven Generation and Evaluation of Discrete-Event World Models via the DEVS Formalism [9.578714623643291]
  We seek a principled middle ground that combines the reliability of explicit simulators with the flexibility of learned models.<n>We advocate explicit, executable discrete-event world models synthesized directly from natural-language specifications.
  arXiv  Detail & Related papers  (2026-03-04T06:50:32Z)
• On Sample-Efficient Generalized Planning via Learned Transition Models [8.508980351550003]
  Generalized planning studies the construction of solution strategies that generalize across families of planning problems sharing a common domain model.<n>Recent Transformer-based planners, such as PlanGPT and Plansformer, largely cast generalized planning as direct action-sequence prediction.<n>We formulate generalized planning as a transition-model learning problem, in which a neural model explicitly approximates the successor-state function $hat approx $ and generates plans by rolling out symbolic state trajectories.
  arXiv  Detail & Related papers  (2026-02-26T16:13:46Z)
• MOTIF: Learning Action Motifs for Few-shot Cross-Embodiment Transfer [55.982504915794514]
  Cross-embodiment policies typically rely on shared-private architectures.<n>We introduce MOTIF for efficient few-shot cross-embodiment transfer.<n>We show that MOTIF significantly outperforms strong baselines in few-shot transfer scenarios.
  arXiv  Detail & Related papers  (2026-02-14T13:21:40Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• Human locomotor control timescales depend on the environmental context and sensory input modality [37.48294298569551]
  We present a unified data-driven framework to quantify the control timescales.<n>We apply this framework across tasks including walking and running.<n>Our framework reveals the factors that influence locomotor foot placement control timescales.
  arXiv  Detail & Related papers  (2025-03-20T16:57:15Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.