The JEPA Predictor: A Transferable Operator for Occluded Feature Completion

• URL: http://arxiv.org/abs/2607.16274v2
• Date: Tue, 21 Jul 2026 17:41:40 GMT
• Title: The JEPA Predictor: A Transferable Operator for Occluded Feature Completion
• Abstract summary: Joint-Embedding Predictive Architectures (JEPAs) train a predictor jointly with their encoder, but downstream deployment discards the predictor and reads features from the encoder alone.<n>We show that this operator is portable across encoder families.
• Score: 1.2664079403656765
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-Embedding Predictive Architectures (JEPAs) train a predictor jointly with their encoder, but downstream deployment discards the predictor and reads features from the encoder alone. The predictor is, by construction, a learned operator from visible-context features to features at masked positions, the structure a partial-view classifier needs. We show that this operator is portable across encoder families. We first establish that, at heavy mask, retaining the frozen predictor on a JEPA encoder substantially closes the accuracy gap against the strongest non-JEPA discriminative baselines. We then bolt the frozen predictors of I-JEPA and V-JEPA 2 onto four non-JEPA hosts (CLIP, DINOv3, DINOv2, MAE) through a single linear projection between feature spaces, fit in closed form on 500 ImageNet-1k images. Across both ImageNet-9 and Stanford Dogs and across three mask fractions, the lift over each host's masked-encoder baseline grows monotonically with the mask fraction K in every host-donor pair. CLIP paired with the I-JEPA predictor recovers most of the accuracy that masking removed on ImageNet-9 at heavy occlusion, and lifts fine-grained Stanford Dogs from 15.9% to 52.1% (+36 pp). The mechanism is identifiable: the projection pays a fixed cost on visible patches and the predictor provides a growing benefit on masked patches; the benefit dominates the heavy-occlusion regime. At low K on fine-grained classification the projection cost exceeds the benefit, defining the boundary where the linear bridge breaks down. The frozen JEPA predictor functions as a portable operator for occluded feature completion across encoder families, requiring no retraining of either model while fitting matched linear probes per mask fraction.

Related papers

• LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics [50.45297537821907]
  We introduce LeVJEPA, the first video encoder trained under LeJEPA's collapse-free objective.<n>A single encoder is trained with an invariance loss over global and local views of a clip.<n>LeVJEPA matches or surpasses V-JEPA 2 across ViT-S/B/L at 5.6 to 20.8x less pretraining compute.
  arXiv  Detail & Related papers  (2026-08-27T17:26:24Z)
• FactorJEPA: Factorizing Monolithic Futures into Layout-Agent-Interaction Channels for Crowded and Chaotic Global South Urban Worlds [20.90457097908955]
  We study a largely unexplored regime: populous, crowded, and chaotic Global South urban environments.<n>Unlike the lower-density, lane-structured settings that dominate existing evaluations, these scenes exhibit soft spatial boundaries.<n>We introduce the first large-scale dataset for this regime: 1,000 hours of drive-through, walk-through, and aerial video across 22 cities.
  arXiv  Detail & Related papers  (2026-08-02T07:32:46Z)
• CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning [3.526682160724688]
  Crop-based Forward JEPA is proposed as an innovative mask-free framework that replaces masking with multi-horizon forward prediction.<n>A strong asymmetry is identified between the online encoder and the exponential moving average (EMA) target encoder, both produced from a single training run.<n> CF-JEPA achieves the highest average accuracy and rank on UCR and UEA among self-supervised baselines.
  arXiv  Detail & Related papers  (2026-06-05T08:21:20Z)
• SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale [54.70985426016736]
  We present SkillDAG, which models inter-skill relationships as a typed directed graph.<n>Each search returns vector matches, typed-edge neighbors, and conflict signals, and a propose-then-commit protocol lets the agent register execution-backed edges.<n>On ALFWorld and SkillsBench with MiniMax-M2.7, SkillDAG reaches 67.1% success and 27.3% reward.
  arXiv  Detail & Related papers  (2026-06-02T02:45:21Z)
• From Measurement to Mitigation: Quantifying and Reducing Identity Leakage in Image Representation Encoders with Linear Subspace Removal [9.492715910999749]
  We propose a benchmark of visual embeddings that reports open-set verification at low false-accept rates and a one-shot linear projector that removes an estimated identity subspace.<n>Across CelebA-20 and VGGFace2, we show that these encoders are robust under open-set linear probes.<n>Our results establish the first attacker-calibrated facial privacy audit of non-FR encoders.
  arXiv  Detail & Related papers  (2026-04-07T01:03:51Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics [53.247652209132376]
  Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D.<n>We present a comprehensive theory of JEPAs and instantiate it in bf LeJEPA, a lean, scalable, and theoretically grounded training objective.
  arXiv  Detail & Related papers  (2025-11-11T18:21:55Z)
• Split-Fuse-Transport: Annotation-Free Saliency via Dual Clustering and Optimal Transport Alignment [10.013879292840707]
  Salient object detection (SOD) aims to segment visually prominent regions in images and serves as a foundational task for various computer vision applications.<n>We posit that SOD can now reach near-supervised accuracy without a single pixel-level label, but only when reliable pseudo-masks are available.<n>We introduce POTNet, an adaptation of Prototypical Optimal Transport that replaces POT's single k-means step with an entropy-guided dual-clustering head.<n>This split-fuse-transport design yields sharper, part-aware pseudo-masks in a single forward pass, without handcrafted priors.
  arXiv  Detail & Related papers  (2025-10-20T12:27:55Z)
• Label-Efficient Grasp Joint Prediction with Point-JEPA [0.0]
  3D self-supervised pretraining with Point--JEPA enables label-efficient grasp joint-angle prediction.<n>JEPA-style pretraining is a practical lever for data-efficient grasp learning.
  arXiv  Detail & Related papers  (2025-09-13T21:00:03Z)
• CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification [65.46685389276443]
  We ground our work on CLIP, a vision-language pre-trained encoder model that can perform zero-shot classification by matching an image with text prompts.<n>We then formulate purification risk as the KL divergence between the joint distributions purification process.<n>We propose two variants for our CLIPure approach: CLI-Diff which models the likelihood of images' latent vectors, and CLIPure-Cos which models the likelihood with the cosine similarity between the embeddings of an image and a photo of a.''
  arXiv  Detail & Related papers  (2025-02-25T13:09:34Z)
• How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks [14.338754598043968]
  Two competing paradigms exist for self-supervised learning of data representations. Joint Embedding Predictive Architecture (JEPA) is a class of architectures in which semantically similar inputs are encoded into representations that are predictive of each other.
  arXiv  Detail & Related papers  (2024-07-03T19:43:12Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.