X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling

• URL: http://arxiv.org/abs/2605.24892v2
• Date: Fri, 29 May 2026 21:54:20 GMT
• Title: X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling
• Abstract summary: We introduce X-Foresight, a predictive world model integrated directly into the Vision-Language-Action architecture.<n>By predicting semantically distant chunks rather than adjacent frames, X-Foresight escapes trivial extrapolation.<n> Comprehensive experiments demonstrate that X-Foresight significantly outperforms VLA baselines in planning performance.
• Score: 47.54820149491433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physical world knowledge resides mainly in videos. Equipping Vision-Language-Action (VLA) models with such knowledge is fundamental for safe and generalizable planning. Predictive world modeling enables VLA to internalize physical dynamics and long-term causality by predicting future video from past observations. However, naive next-frame prediction faces two challenges: 1) unlike semantically distinct text tokens, video tokens are low-entropy and redundant, causing prediction to degenerate into trivial extrapolation. 2) world modeling poses a temporal dilemma: dense prediction captures instantaneous dynamics, but cannot efficiently model long-horizon causality. To learn world knowledge effectively, we introduce X-Foresight, a predictive world model integrated directly into the VLA architecture to jointly learn world modeling and real-time action control. At its core lies a long-horizon chunk-wise auto-regressive strategy that addresses both challenges: by predicting semantically distant chunks rather than adjacent frames, it escapes trivial extrapolation, while preserving dense intra-chunk frames for instantaneous dynamics and sparse inter-chunk transitions for long-term causality. A curriculum learning schedule progressively extends prediction horizons and stabilizes long-horizon training. To capture long-term causality effectively, we present temporal importance sampling, which concentrates supervision on safety-critical chunks identified by ego-motion and behavioral signals. We further delegate photorealistic synthesis to a diffusion-based multi-view renderer, improving photorealistic appearance. Comprehensive experiments demonstrate that X-Foresight significantly outperforms VLA baselines in planning performance while maintaining strong generative fidelity, establishing a robust paradigm for world-knowledge-driven autonomous systems.

Related papers

• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow [26.147123396344128]
  We introduce Physical-Time Flow (textbfPT-Flow), a novel approach that learns a continuous latent velocity field operating in physical time.<n>Under PT-Flow, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space.<n>We construct textbfODEWorld, a continuous-time latent world model that is both efficient and versatile.
  arXiv  Detail & Related papers  (2026-07-30T09:37:30Z)
• GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving [56.50032398571235]
  GraphWorld is an E2E-AD framework that explicitly enhances long-horizon planning through latent world modeling.<n>We present a World-State-Conditioned Planning that learns ego-centric latent world representations by modeling interactions between an ego vehicle and surrounding agents.<n>This latent world state captures key interaction dynamics and safety-relevant semantics, and serves as a conditioning signal to guide long-horizon, safety-aware trajectory planning.
  arXiv  Detail & Related papers  (2026-06-15T06:26:46Z)
• Learning Vision-Language-Action World Models for Autonomous Driving [15.103497388527943]
  Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving.<n>VLA models often lack explicit modeling of temporal dynamics and global world consistency.<n>We present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning.
  arXiv  Detail & Related papers  (2026-04-10T07:38:05Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• Chain of World: World Model Thinking in Latent Motion [24.24061036481793]
  Vision-Language-Action (VLA) models often overlook the predictive and temporal-causal structure underlying visual dynamics.<n>We introduce CoWVLA, a new "Chain of World" paradigm that unifies world-model temporal reasoning with a disentangled latent motion representation.<n>CoWVLA outperforms existing world-model and latent-action approaches and achieves moderate computational efficiency.
  arXiv  Detail & Related papers  (2026-03-03T17:52:06Z)
• PAN: A World Model for General, Interactable, and Long-Horizon World Simulation [49.805071498152536]
  We introduce PAN, a general, interactable, and long-horizon world model.<n>It predicts future world states through high-quality video simulation conditioned on history and natural language actions.<n>Experiments show that PAN achieves strong performance in action-conditioned world simulation, long-horizon forecasting, and simulative reasoning.
  arXiv  Detail & Related papers  (2025-11-12T07:20:35Z)
• Learning Primitive Embodied World Models: Towards Scalable Robotic Learning [50.32986780156215]
  We propose a novel paradigm for world modeling--Primitive Embodied World Models (PEWM)<n>By restricting video generation to fixed short horizons, our approach enables fine-grained alignment between linguistic concepts and visual representations of robotic actions.<n>Our framework bridges the gap between fine-grained physical interaction and high-level reasoning, paving the way toward scalable, interpretable, and general-purpose embodied intelligence.
  arXiv  Detail & Related papers  (2025-08-28T14:31:48Z)
• DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge [41.030494146004806]
  We propose DreamVLA, a novel VLA framework that integrates comprehensive world knowledge forecasting to enable inverse dynamics modeling.<n>DreamVLA introduces a dynamic-region-guided world knowledge prediction, integrated with the spatial and semantic cues, which provide compact yet comprehensive representations for action planning.<n>Experiments on both real-world and simulation environments demonstrate that DreamVLA achieves 76.7% success rate on real robot tasks.
  arXiv  Detail & Related papers  (2025-07-06T16:14:29Z)
• DeepVerse: 4D Autoregressive Video Generation as a World Model [16.877309608945566]
  We introduce DeepVerse, a novel 4D interactive world model explicitly incorporating geometric predictions from previous timesteps into current predictions on actions.<n> Experiments demonstrate that by incorporating explicit geometric constraints, DeepVerse captures richer-temporal relationships and underlying physical dynamics.<n>This capability significantly reduces drift and enhances temporal consistency, enabling the model to reliably generate extended future sequences.
  arXiv  Detail & Related papers  (2025-06-01T17:58:36Z)
• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving [19.81442567260658]
  We propose a visual-temporalT framework that enables VLAs to think in images.<n>On nuScenes and NAVSIM, FSDrive improves accuracy and reduces collisions.
  arXiv  Detail & Related papers  (2025-05-23T09:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.