FACT: Failure-Aware Causal Training for World-Action Models

• URL: http://arxiv.org/abs/2608.10232v1
• Date: Mon, 10 Aug 2026 21:10:46 GMT
• Title: FACT: Failure-Aware Causal Training for World-Action Models
• Abstract summary: Recent world-action models show that co-training policies with future prediction can provide physical priors for action generation.<n>We introduce FACT, a causal World-Action Model that predicts future video and task progress conditioned on the executed action.
• Score: 20.123306787874828
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation. Building on the future-prediction ability of video models, many WAMs generate future videos and recover actions with inverse-dynamics models, or use these predicted videos as goal conditions for action generation. In both cases, the world model is trained mostly on successful demonstrations and has little reason to predict the consequences of bad actions. We introduce FACT, a causal World-Action Model that predicts future video and task progress conditioned on the executed action. This action-conditioned interface allows failure rollouts to supervise action consequences, turning bad actions into valid future targets rather than being discarded. Failure-aware training makes the progress predictor aware of both successful and failed action outcomes, which can optionally be used to score sampled action candidates at inference. Extensive experiments on simulation and real-world bimanual manipulation tasks show that FACT outperforms many existing baselines, improves as failure data are incorporated into training, and reduces success-biased future hallucination under bad actions. See more details at https://fact-wam.github.io/

Related papers

• Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning [53.77164209208635]
  We introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and trajectory alignment.<n>We propose WorldSync, which strengthens action following along three axes: distributional coverage, representational grounding, and intervention-effect alignment.<n>Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement.
  arXiv  Detail & Related papers  (2026-08-25T17:59:49Z)
• DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation [67.08835970838996]
  Reinforcement learning holds great promise for improving robot policies beyond the limits of imitation learning.<n>Two key challenges remain: acquiring diverse failure data at scale and obtaining fine-grained reward signals beyond sparse trajectory-level success labels.<n>We introduce DenseReward, a dense robotic reward model that addresses both challenges.
  arXiv  Detail & Related papers  (2026-07-14T17:59:29Z)
• From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data [71.22409934108924]
  Human videos are abundant and capture rich interactions, providing diverse semantic and physical cues for real-world manipulation.<n>This survey provides a unified view of how human videos are transformed into effective knowledge for Vision-Language-Action (VLA) models.<n>We highlight three key open challenges in this area: structuring unstructured videos into training-ready episodes, grounding video-derived supervision into robot-executable actions under embodiment and viewpoint heterogeneity, and designing evaluation protocols that better predict real-world deployment performance and transfer efficiency.
  arXiv  Detail & Related papers  (2026-05-18T06:19:16Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• Learning Lifted Action Models from Unsupervised Visual Traces [38.479081137300945]
  We propose a deep learning framework that jointly learns state prediction, action prediction, and a lifted action model.<n>We also introduce a mixed-integer linear program (MILP) to prevent prediction collapse and self-reinforcing errors.<n>Experiments across multiple domains show that integrating MILP-based correction helps the model escape local optima and converge toward globally consistent solutions.
  arXiv  Detail & Related papers  (2026-04-21T03:49:04Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• World Guidance: World Modeling in Condition Space for Action Generation [39.098315503589895]
  Leveraging future observation modeling to facilitate action generation presents a promising avenue for enhancing the capabilities of Vision-Language-Action (VLA) models.<n>We propose WoG, a framework that maps future observations into compact conditions by injecting them into the action inference pipeline.<n>We demonstrate that modeling and predicting this condition space not only facilitates fine-grained action generation but also exhibits superior generalization capabilities.
  arXiv  Detail & Related papers  (2026-02-25T15:27:09Z)
• Ego-centric Predictive Model Conditioned on Hand Trajectories [52.531681772560724]
  In egocentric scenarios, anticipating both the next action and its visual outcome is essential for understanding human-object interactions.<n>We propose a unified two-stage predictive framework that jointly models action and visual future in egocentric scenarios.<n>Our approach is the first unified model designed to handle both egocentric human activity understanding and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-08-27T13:09:55Z)
• DiffAnt: Diffusion Models for Action Anticipation [12.022815981853071]
  Anticipating future actions is inherently uncertain. Given an observed video segment containing ongoing actions, multiple subsequent actions can plausibly follow. In this work, we rethink action anticipation from a generative view, employing diffusion models to capture different possible future actions. Our code and trained models will be published on GitHub.
  arXiv  Detail & Related papers  (2023-11-27T16:40:09Z)
• Learning the Effects of Physical Actions in a Multi-modal Environment [17.757831697284498]
  Large Language Models (LLMs) handle physical commonsense information inadequately. We introduce the multi-modal task of predicting the outcomes of actions solely from realistic sensory inputs. We show that multi-modal models can capture physical commonsense when augmented with visual information.
  arXiv  Detail & Related papers  (2023-01-27T16:49:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.