Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency

• URL: http://arxiv.org/abs/2608.12939v1
• Date: Thu, 13 Aug 2026 08:18:41 GMT
• Title: Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency
• Abstract summary: We introduce a diagnostic that measures how far a clean history and a visually perturbed view of it diverge after being rolled forward.<n>We prove that this divergence bounds the perturbation-induced change in multi-step prediction error and planner cost.<n>Experiments on four visual control tasks show that pairwise ACPC predicts perturbation-induced prediction and cost changes.
• Score: 38.88857369304745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-embedding predictive architectures (JEPAs) learn world models that predict in a compact latent space rather than in pixels, reducing the pressure to model nuisance appearance. Yet this provides no guarantee against visual perturbations: they can still alter the encoded representation and affect subsequent action-conditioned predictions. Bisimulation captures this requirement precisely: two observations should be treated as the same state only when their action-conditioned consequences agree. Guided by this criterion, we introduce Action-Conditioned Predictive Consistency (ACPC), a diagnostic that measures how far a clean history and a visually perturbed view of it diverge after being rolled forward under the same action sequence. We prove that this divergence bounds the perturbation-induced change in multi-step prediction error and planner cost. Building on pairwise ACPC, we define two complementary measures: the Invariance Radius (IR) summarizes clean-perturbed rollout spread, while the Separation Rate (SR) checks whether different states remain distinguishable after rollout. Experiments on four visual control tasks show that pairwise ACPC predicts perturbation-induced prediction and cost changes. On LeWM, the IR-SR screen transfers across tasks, and the joint diagnostic remains informative under blur and resize. PLDM exhibits similar diagnostic trends under a different architecture.

Related papers

• Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models [62.522141464687195]
  We introduce Action-JND, which extends the principle of just noticeable difference (JND) to embodied perception.<n>A token change is considered tolerable only when the induced action deviation remains within a tolerated margin.<n>The resulting action-tolerance score serves as a plug-and-play criterion for VLA compression paradigms.
  arXiv  Detail & Related papers  (2026-08-21T15:59:37Z)
• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• Amortized Interventional Forecasting for Multivariate CIR Processes [2.918919242459813]
  Mean-reverting dynamics are pervasive in finance.<n>CIR models capture only emphcorrelated co-movement, not emphcausal influence between series.
  arXiv  Detail & Related papers  (2026-08-04T14:17:35Z)
• On the Identifiability of Controlled World Models [34.31084112465058]
  This paper presents a joint identifiability condition for controlled world models with Gaussian latent states.<n>We prove that when this condition holds, minimizing the LeJEPA-style predictive objective can recover both latent states and controlled dynamics.<n>We also characterize an attainable amplification of counterfactual prediction error that scales inversely with the weakest conditional action-excitation margin.
  arXiv  Detail & Related papers  (2026-07-24T15:49:12Z)
• DCS: A Unified Conditional Sensitivity Framework for Cross-Modal Copyright Infringement Detection [0.0]
  We develop a unified post-hoc detection framework that treats copyright infringement evidence as a counterfactual conditional distribution shift.<n>A protected target is suspicious when the model's behavior under aligned conditions would change measurably if that target were included in, or removed from, the training process.<n>The framework is instantiated for ridge-regularized linear regression, conditional diffusion models, autoregressive language models, and multimodal models.
  arXiv  Detail & Related papers  (2026-07-24T07:00:44Z)
• Predictive Objectives Discard Exogenous Control-Relevant Features: A Controlled Mechanistic Study [0.0]
  Joint-embedding predictive (JEPA) objectives learn representations by predicting future latents.<n>This occurs because the objective optimize temporal predictability rather than control-relevance.<n>We show that as little as 2% of reward-labeled transitions recovers the feature, the effect holds across two environments.
  arXiv  Detail & Related papers  (2026-06-29T10:00:50Z)
• Perturb and Correct: Post-Hoc Ensembles using Affine Redundancy [0.0]
  We introduce Perturb-and-Correct (P&C), a method for constructing diverse predictors from a single pretrained network.<n>P&C applies random hidden layer perturbations with a least-squares correction in the subsequent affine layer, producing predictors that agree on calibration data while remaining free to disagree away from it.<n> Empirically, P&C achieves a strong ID/OOD tradeoff across MuJoCo dynamics prediction and CIFAR-10 OOD detection, matching or outperforming standard post-hoc baselines while requiring only a single pretrained model.
  arXiv  Detail & Related papers  (2026-05-02T22:48:11Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• BURN: Backdoor Unlearning via Adversarial Boundary Analysis [73.14147934175604]
  Backdoor unlearning aims to remove backdoor-related information while preserving the model's original functionality.<n>We propose Backdoor Unlearning via adversaRial bouNdary analysis (BURN), a novel defense framework that integrates false correlation decoupling, progressive data refinement, and model purification.
  arXiv  Detail & Related papers  (2025-07-14T17:13:06Z)
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis [55.13545823385091]
  Federated reinforcement learning (FedRL) enables collaborative learning while preserving data privacy by preventing direct data exchange between agents.<n>In real-world applications, each agent may experience slightly different transition dynamics, leading to inherent model mismatches.<n>We show that even moderate levels of information sharing significantly mitigate environment-specific errors.
  arXiv  Detail & Related papers  (2025-03-21T18:06:28Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.