Predictive Objectives Discard Exogenous Control-Relevant Features: A Controlled Mechanistic Study

• URL: http://arxiv.org/abs/2606.30068v1
• Date: Mon, 29 Jun 2026 10:00:50 GMT
• Title: Predictive Objectives Discard Exogenous Control-Relevant Features: A Controlled Mechanistic Study
• Abstract summary: Joint-embedding predictive (JEPA) objectives learn representations by predicting future latents.<n>This occurs because the objective optimize temporal predictability rather than control-relevance.<n>We show that as little as 2% of reward-labeled transitions recovers the feature, the effect holds across two environments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-embedding predictive (JEPA-style) objectives learn representations by predicting future latents. In doing so they can discard features that are exogenous (uncontrollable by the agent) yet control-relevant, even when those features are trivially encodable. This occurs because the objective optimizes temporal predictability rather than control-relevance. We isolate this failure mode in a controlled 2x2 experimental design that varies feature controllability and relevance independently, using a predictability knob that decouples a feature's temporal predictability from its control-relevance. Comparing six objectives: reconstruction, JEPA, action-conditioned JEPA, controllability-based JEPA, inverse dynamics under a random policy, and reward-grounded JEPA, we observe that all evaluated reward-free predictive objectives leave the exogenous control-relevant feature near chance accuracy, while a reward-grounded variant retains it selectively. The remedy is label-efficient and robust: as little as 2% of reward-labeled transitions recovers the feature, the effect holds across two environments with different surface forms, and it persists across latent dimensions from 16 to 1024. Comparing the learned latent geometry against bisimulation theory's prediction, the JEPA latent realizes only a small fraction of the class separation a supervised reference attains.

Related papers

• JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics [9.006010556484968]
  In self-predictive models, the encoder and predictor are optimized jointly and can co-adapt to latent transitions.<n>We introduce the cross-predictive JEPA (JEPA-x), which grounds latent dynamics in privileged physical trajectories.
  arXiv  Detail & Related papers  (2026-08-25T04:07:07Z)
• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency [38.88857369304745]
  We introduce a diagnostic that measures how far a clean history and a visually perturbed view of it diverge after being rolled forward.<n>We prove that this divergence bounds the perturbation-induced change in multi-step prediction error and planner cost.<n>Experiments on four visual control tasks show that pairwise ACPC predicts perturbation-induced prediction and cost changes.
  arXiv  Detail & Related papers  (2026-08-13T08:18:41Z)
• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• Calibrating Conservatism for Scalable Oversight [9.553819152637493]
  We introduce Calibrated Collective Oversight (CCO), which aggregates diverse auxiliary scoring functions into a penalty measuring deviation from a conservative baseline.<n>CCO calibrates this conservatism online using Conformal Decision Theory, ensuring that undesirable outcomes remain below a user-specified target threshold.<n>On a modified version of SWE-bench, weaker overseers successfully constrain an adversarially misaligned stronger agent; on MACHIAVELLI, CCO substantially reduces ethical violations while preserving reward.
  arXiv  Detail & Related papers  (2026-05-27T17:56:47Z)
• UWM-JEPA: Predictive World Models That Imagine in Belief Space [0.2864713389096699]
  We introduce the Unitary World Model JEPA, a JEPA world model with a density-matrix latent on a joint system-environment space and a learned unitary predictor.<n>The construction preserves the joint-state spectrum exactly during rollout, so the predictor itself cannot dissipate the represented uncertainty.<n>For JEPA world models to imagine under partial observability, latent geometry and predictor dynamics matter, not frozen context-encoding capacity alone.
  arXiv  Detail & Related papers  (2026-05-25T00:28:51Z)
• Var-JEPA: A Variational Formulation of the Joint-Embedding Predictive Architecture -- Bridging Predictive and Generative Self-Supervised Learning [2.1127261244588156]
  Joint-Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning.<n>We argue that the canonical JEPA design, coupled encoders with a context-to-target predictor, mirrors the variational posteriors and learned conditional priors.<n>Var-JEPA makes the latent generative structure explicit by optimizing a single Evidence Lower Bound.
  arXiv  Detail & Related papers  (2026-03-20T16:36:29Z)
• Observationally Informed Adaptive Causal Experimental Design [55.998153710215654]
  We propose Active Residual Learning, a new paradigm that leverages the observational model as a foundational prior.<n>This approach shifts the experimental focus from learning target causal quantities from scratch to efficiently estimating the residuals required to correct observational bias.<n> Experiments on synthetic and semi-synthetic benchmarks demonstrate that R-Design significantly outperforms baselines.
  arXiv  Detail & Related papers  (2026-03-04T06:52:37Z)
• DiffusionRollout: Uncertainty-Aware Rollout Planning in Long-Horizon PDE Solving [28.440795270548705]
  DiffusionRollout is a novel selective rollout planning strategy for autoregressive diffusion models.<n>We introduce a mechanism that adaptively selects step sizes during autoregressive rollouts, improving long-term prediction reliability.
  arXiv  Detail & Related papers  (2026-02-14T06:08:05Z)
• VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models [0.0]
  We introduce emphVariational JEPA (VJEPA), a textitprobabilistic generalization that learns a predictive distribution over future latent states via a variational objective.<n>VJEPA representations can serve as sufficient information states for optimal control without pixel reconstruction, while providing formal guarantees for collapse avoidance.<n>We propose emphBayesian JEPA (BJEPA), an extension that factorizes the predictive belief into a learned dynamics expert and a modular prior expert.
  arXiv  Detail & Related papers  (2026-01-20T18:04:16Z)
• Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models [60.728124907335]
  This work introduces Weather Adaptive Adversarial Perturbation Optimization (WAAPO), a novel framework for generating targeted adversarial perturbations.<n>WAAPO achieves this by incorporating constraints for channel sparsity, spatial localization, and smoothness, ensuring that perturbations remain physically realistic and imperceptible.<n>Our experiments highlight critical vulnerabilities in AI-driven forecasting models, where small perturbations to initial conditions can result in significant deviations.
  arXiv  Detail & Related papers  (2025-12-09T17:20:56Z)
• Efficient Thought Space Exploration through Strategic Intervention [54.35208611253168]
  We propose a novel Hint-Practice Reasoning (HPR) framework that operationalizes this insight through two synergistic components.<n>The framework's core innovation lies in Distributional Inconsistency Reduction (DIR), which dynamically identifies intervention points.<n> Experiments across arithmetic and commonsense reasoning benchmarks demonstrate HPR's state-of-the-art efficiency-accuracy tradeoffs.
  arXiv  Detail & Related papers  (2025-11-13T07:26:01Z)
• SynCast: Synergizing Contradictions in Precipitation Nowcasting via Diffusion Sequential Preference Optimization [62.958457694151384]
  We introduce preference optimization into precipitation nowcasting for the first time, motivated by the success of reinforcement learning from human feedback in large language models.<n>In the first stage, the framework focuses on reducing FAR, training the model to effectively suppress false alarms.
  arXiv  Detail & Related papers  (2025-10-22T16:11:22Z)
• TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning [63.73629127832652]
  We introduce TD-JEPA, which leverages TD-based latent-predictive representations into unsupervised RL.<n> TD-JEPA trains explicit state and task encoders, a policy-conditioned multi-step predictor, and a set of parameterized policies directly in latent space.<n> Empirically, TD-JEPA matches or outperforms state-of-the-art baselines on locomotion, navigation, and manipulation tasks across 13 datasets.
  arXiv  Detail & Related papers  (2025-10-01T10:21:18Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.