Flow-JEPA: Flow Matching for Robust Latent Dynamics in JEPA World Models

• URL: http://arxiv.org/abs/2608.29029v1
• Date: Sat, 29 Aug 2026 03:40:06 GMT
• Title: Flow-JEPA: Flow Matching for Robust Latent Dynamics in JEPA World Models
• Abstract summary: Joint-Embedding Predictive Architectures (JEPAs) have shown strong potential for learning compact predictive representations.<n>We propose Flow-JEPA, a conditional flow matching dynamics model that jointly generates a sequence of future latent states conditioned on the current observation and actions.
• Score: 18.035166689862205
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-Embedding Predictive Architectures (JEPAs) have shown strong potential for learning compact predictive representations, and LeWorldModel (LeWM) extends this paradigm to reconstruction-free latent world modeling from pixels. However, its deterministic autoregressive predictor generates future states through repeated one-step transitions, which can accumulate errors and remain sensitive to task-irrelevant visual perturbations. In this work, we propose Flow-JEPA (F-JEPA), a conditional flow matching dynamics model that jointly generates a sequence of future latent states conditioned on the current observation and actions. A Gaussian distribution serves as the flow source, exposing the vector field to perturbed latent trajectories as it learns to transport them toward clean future representations. This formulation retains the reconstruction-free JEPA framework while replacing point-wise transition regression with stochastic trajectory-level prediction. F-JEPA raises mean success from $86\%$ to $92\%$ under clean observations and from $67\%$ to $86\%$ under noisy conditions, suggesting that conditional flow matching provides a promising alternative to deterministic autoregressive dynamics in JEPA world models.

Related papers

• JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics [9.006010556484968]
  In self-predictive models, the encoder and predictor are optimized jointly and can co-adapt to latent transitions.<n>We introduce the cross-predictive JEPA (JEPA-x), which grounds latent dynamics in privileged physical trajectories.
  arXiv  Detail & Related papers  (2026-08-25T04:07:07Z)
• WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving [8.510409798128272]
  We present WA-JEPA, a VEPA-native world-action model for autonomous driving planning.<n>WA-JEPA employs hybrid future-masked pre-training, where the model infers future latents from observed context.<n>Results: WA-JEPA reaches 91.7S on NAVSIM, surpassing the strongest end-to-end and world-action baselines.
  arXiv  Detail & Related papers  (2026-08-21T10:54:35Z)
• Perceptual Flow Matching for Few-Step Generative Modeling [64.26537634881244]
  Perceptual Flow Matching is a framework for few-step generation in flow-matching models.<n>We show that PFM produces high-quality few-step generators when supervised in an appropriate representation space.
  arXiv  Detail & Related papers  (2026-07-03T17:55:01Z)
• Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models [26.928611870426383]
  JointEmbedding Predictive Architectures (JEPAs) provide a simple framework for learning world models by predicting future latent representations.<n>The recent LeWorldModel (LeWM) shows that this issue can be alleviated bysimply constraining latent embeddings with an isotropic Gaussian prior.<n>We propose ame, which seeks a favorable operatingpoint on the bias-variance frontier by applying Gaussian constraints inmultiple random subspaces rather than in the originalembedding space.
  arXiv  Detail & Related papers  (2026-05-10T00:51:47Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models [0.0]
  We introduce emphVariational JEPA (VJEPA), a textitprobabilistic generalization that learns a predictive distribution over future latent states via a variational objective.<n>VJEPA representations can serve as sufficient information states for optimal control without pixel reconstruction, while providing formal guarantees for collapse avoidance.<n>We propose emphBayesian JEPA (BJEPA), an extension that factorizes the predictive belief into a learned dynamics expert and a modular prior expert.
  arXiv  Detail & Related papers  (2026-01-20T18:04:16Z)
• Elucidated Rolling Diffusion Models for Probabilistic Weather Forecasting [52.6508222408558]
  We introduce Elucidated Rolling Diffusion Models (ERDM)<n>ERDM is the first framework to unify a rolling forecast structure with the principled, performant design of Elucidated Diffusion Models (EDM)<n>On 2D Navier-Stokes simulations and ERA5 global weather forecasting at 1.5circ resolution, ERDM consistently outperforms key diffusion-based baselines.
  arXiv  Detail & Related papers  (2025-06-24T21:44:31Z)
• Probabilistic Forecasting via Autoregressive Flow Matching [1.5467259918426441]
  FlowTime is a generative model for probabilistic forecasting of timeseries data.<n>We decompose the joint distribution of future observations into a sequence of conditional densities, each modeled via a shared flow.<n>We demonstrate the effectiveness of FlowTime on multiple dynamical systems and real-world forecasting tasks.
  arXiv  Detail & Related papers  (2025-03-13T13:54:24Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL [50.385005413810084]
  Dyna-style model-based reinforcement learning contains two phases: model rollouts to generate sample for policy learning and real environment exploration. $textttCOPlanner$ is a planning-driven framework for model-based methods to address the inaccurately learned dynamics model problem.
  arXiv  Detail & Related papers  (2023-10-11T06:10:07Z)
• Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization [60.73540999409032]
  We show that expressive autoregressive dynamics models generate different dimensions of the next state and reward sequentially conditioned on previous dimensions. We also show that autoregressive dynamics models are useful for offline policy optimization by serving as a way to enrich the replay buffer.
  arXiv  Detail & Related papers  (2021-04-28T16:48:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.