BiJEPA: Bi-directional Joint Embedding Predictive Architecture for Symmetric Representation Learning

• URL: http://arxiv.org/abs/2603.00049v1
• Date: Tue, 10 Feb 2026 01:16:20 GMT
• Title: BiJEPA: Bi-directional Joint Embedding Predictive Architecture for Symmetric Representation Learning
• Abstract summary: BiJEPA enforces cycle-consistent predictability between data segments.<n>We evaluate BiJEPA on three distinct modalities: synthetic periodic signals, chaotic Lorenz attractor trajectories, and high-dimensional image data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-Supervised Learning (SSL) has shifted from pixel-level reconstruction to latent space prediction, spearheaded by the Joint Embedding Predictive Architecture (JEPA). While effective, standard JEPA models typically rely on a uni-directional prediction mechanism (e.g. Context $\to$ Target), potentially neglecting the informative signal inherent in the inverse relationship, degrading its performance. In this work, we propose \textbf{BiJEPA}, a \textit{Bi-Directional Joint Embedding Predictive Architecture} that enforces cycle-consistent predictability between data segments. We address the inherent instability of symmetric prediction (representation explosion) by introducing a critical norm regularization mechanism on the representation vectors. We evaluate BiJEPA on three distinct modalities: synthetic periodic signals, chaotic Lorenz attractor trajectories, and high-dimensional image data (MNIST). Our results demonstrate that BiJEPA achieves stable convergence without collapse, captures the semantic structure of chaotic systems, and learns robust temporal and spatial representations capable of generation and generalisation, offering a more holistic approach to representation learning.

Related papers

• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• APO: Unsupervised Atomic Policy Optimization for 3D Structure Prediction of Atomic Systems [64.56252173115246]
  We propose (Atomic Policy Optimization) a fully unsupervised alignment framework that eliminates the need for ground-truth reference structures.<n>APO adapts group-relative policy optimization to 3D atomic environments, utilizing a novel dual-reward mechanism.<n>Our results suggest that intrinsic physical consistency can serve as a superior guide for alignment compared to noisy, supervised coordinate matching.
  arXiv  Detail & Related papers  (2026-07-30T17:21:58Z)
• HQ-JEPA: Hybrid Quantum Joint-Embedding Predictive Architecture for Cross-Modal Remote Sensing Representation Learning [17.65267962254814]
  HQ-JEPA is a hybrid quantum-classical joint-embedding predictive architecture for cross-modal remote sensing representation learning.<n>Results show that HQ-JEPA achieves competitive and often superior performance over strong self-supervised and remote sensing foundation-model baselines.
  arXiv  Detail & Related papers  (2026-05-29T09:37:35Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• Curvature-Aware PCA with Geodesic Tangent Space Aggregation for Semi-Supervised Learning [52.452902154360565]
  GTSA-PCA is a geometric extension of Principal Component Analysis.<n>It integrates curvature awareness and geodesic consistency within a unified spectral framework.<n>Our results position GTSA-PCA as a principled bridge between statistical and geometric approaches to dimensionality reduction.
  arXiv  Detail & Related papers  (2026-04-20T20:36:36Z)
• Var-JEPA: A Variational Formulation of the Joint-Embedding Predictive Architecture -- Bridging Predictive and Generative Self-Supervised Learning [2.1127261244588156]
  Joint-Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning.<n>We argue that the canonical JEPA design, coupled encoders with a context-to-target predictor, mirrors the variational posteriors and learned conditional priors.<n>Var-JEPA makes the latent generative structure explicit by optimizing a single Evidence Lower Bound.
  arXiv  Detail & Related papers  (2026-03-20T16:36:29Z)
• Invariance on Manifolds: Understanding Robust Visual Representations for Place Recognition [19.200074425090595]
  We propose a Second-Order Geometric Statistics framework that inherently captures geometric stability without training.<n>Our approach introduces a training-free framework built upon fixed, pre-trained backbones, achieving strong zero-shot generalization without parameter updates.
  arXiv  Detail & Related papers  (2026-01-31T18:12:29Z)
• VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models [0.0]
  We introduce emphVariational JEPA (VJEPA), a textitprobabilistic generalization that learns a predictive distribution over future latent states via a variational objective.<n>VJEPA representations can serve as sufficient information states for optimal control without pixel reconstruction, while providing formal guarantees for collapse avoidance.<n>We propose emphBayesian JEPA (BJEPA), an extension that factorizes the predictive belief into a learned dynamics expert and a modular prior expert.
  arXiv  Detail & Related papers  (2026-01-20T18:04:16Z)
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics [53.247652209132376]
  Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D.<n>We present a comprehensive theory of JEPAs and instantiate it in bf LeJEPA, a lean, scalable, and theoretically grounded training objective.
  arXiv  Detail & Related papers  (2025-11-11T18:21:55Z)
• ACT-JEPA: Novel Joint-Embedding Predictive Architecture for Efficient Policy Representation Learning [90.41852663775086]
  ACT-JEPA is a novel architecture that integrates imitation learning and self-supervised learning.<n>We train a policy to predict action sequences and abstract observation sequences.<n>Our experiments show that ACT-JEPA improves the quality of representations by learning temporal environment dynamics.
  arXiv  Detail & Related papers  (2025-01-24T16:41:41Z)
• Graph-level Representation Learning with Joint-Embedding Predictive Architectures [43.89120279424267]
  Joint-Embedding Predictive Architectures (JEPAs) have emerged as a novel and powerful technique for self-supervised representation learning.<n>We show that graph-level representations can be effectively modeled using this paradigm by proposing a Graph Joint-Embedding Predictive Architecture (Graph-JEPA)<n>In particular, we employ masked modeling and focus on predicting the latent representations of masked subgraphs starting from the latent representation of a context subgraph.
  arXiv  Detail & Related papers  (2023-09-27T20:42:02Z)
• DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction [72.20024514713633]
  We present a novel graph neural network (GNN) supervision and prediction strategy DR-Label. The strategy enhances the supervision signal, reduces the multiplicity of solutions in edge representation, and encourages the model to provide node predictions robust. DR-Label was applied to three radically distinct models, each of which displayed consistent performance enhancements.
  arXiv  Detail & Related papers  (2023-03-06T04:01:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.