CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning

• URL: http://arxiv.org/abs/2606.07031v1
• Date: Fri, 05 Jun 2026 08:21:20 GMT
• Title: CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning
• Abstract summary: Crop-based Forward JEPA is proposed as an innovative mask-free framework that replaces masking with multi-horizon forward prediction.<n>A strong asymmetry is identified between the online encoder and the exponential moving average (EMA) target encoder, both produced from a single training run.<n> CF-JEPA achieves the highest average accuracy and rank on UCR and UEA among self-supervised baselines.
• Score: 3.526682160724688
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised learning (SSL) for time-series representation learning is dominated by two paradigms: contrastive methods, which face challenges in constructing positive or negative pairs, and masking-based methods, which disrupt the temporal continuity of time-series signals. Joint-Embedding Predictive Architecture (JEPA) offers a promising alternative by predicting in representation space rather than reconstructing raw inputs. However, existing time-series JEPA variants still rely on masking and therefore inherit its continuity problem. Crop-based Forward JEPA (CF-JEPA) is proposed as an innovative mask-free framework that replaces masking with multi-horizon forward prediction: random crops serve as context views, and short-, mid-, and long-horizon future representations are predicted in the forward temporal direction, directly leveraging the inherent temporal ordering of time-series data as a learning signal. A strong asymmetry is also identified between the online encoder and the exponential moving average (EMA) target encoder, both produced from a single training run: the online encoder develops higher-rank discriminative features, while the EMA target encoder develops smoother, lower-rank temporal features. Exploiting this asymmetry, classification is routed to the online encoder and forecasting or anomaly detection to the EMA target encoder, achieving a 27% reduction in multivariate forecasting mean squared error (MSE) at no additional training cost. Across 126 University of California, Riverside (UCR) and 26 University of East Anglia (UEA) classification datasets, eight electricity transformer temperature forecasting benchmarks, and Key Performance Indicator /Yahoo anomaly detection, CF-JEPA achieves the highest average accuracy and rank on UCR and UEA among self-supervised baselines and ranks second on univariate forecasting and k-nearest neighbors-scored anomaly detection.

Related papers

• Large-Small Model Collaboration for Zero-Shot Surgical Phase Recognition [45.191945465591374]
  Task-specific lightweight models for surgical phase recognition excel at capturing temporal dynamics but generalize poorly under domain shift.<n>We propose textbfLarge-textbfSmall textbfTemporal adaptation (textbfLaST), a novel large-small collaborative framework that enables zero-shot adaptation to unseen clinical domains.
  arXiv  Detail & Related papers  (2026-08-24T07:06:52Z)
• FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales [65.4821703903285]
  FLORO is a multimodal foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus.<n> FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and UAV-derived data.<n>We evaluated FLORO on the PANGAEA benchmark under a frozen-encoder protocol across scene classification, segmentation, and regression tasks.
  arXiv  Detail & Related papers  (2026-05-27T08:55:54Z)
• HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series [1.592463731034439]
  We introduce HEPA (Horizon-conditioned Event Predictive Architecture), built on two key principles.<n>A causal Transformer encoder is pretrained via a Joint-Embedding Predictive Architecture (JEPA)<n>We freeze the encoder and finetune only the predictor toward the target event, producing a monotonic survival cumulative distribution function (CDF) over horizons.
  arXiv  Detail & Related papers  (2026-05-11T18:38:03Z)
• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection [1.7751300245073598]
  AxonAD is an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process.<n>A tail-aggregated query mismatch score measures cosine deviation between predicted and target queries on recent timesteps.<n>On in-vehicle telemetry with interval annotations, AxonAD improves ranking quality and temporal localization over strong baselines.
  arXiv  Detail & Related papers  (2026-03-13T11:40:51Z)
• Advancing accelerator virtual beam diagnostics through latent evolution modeling: an integrated solution to forward, inverse, tuning, and UQ problems [46.568487965999225]
  We propose Latent Evolution Model (LEM), a hybrid machine learning framework with an autoencoder that projects high-dimensional phase spaces into lower-dimensional representations, and transformers to learn temporal dynamics in the latent space.<n>For textitforward modeling, a CVAE encodes 15 unique projections of the 6D phase space into a latent representation, while a transformer predicts downstream latent states from upstream inputs.<n>For textitinverse problems, we address two distinct challenges: (a) predicting upstream phase spaces from downstream observations, and (b) estimating RF settings from the latent space of
  arXiv  Detail & Related papers  (2026-02-26T04:46:26Z)
• Real-Time Proactive Anomaly Detection via Forward and Backward Forecast Modeling [0.0]
  We introduce two proactive anomaly detection frameworks: the Forward Forecasting Model (FFM) and the Backward Reconstruction Model (BRM)<n>FFM forecasts future sequences to anticipate disruptions, while BRM reconstructs recent history from future context to uncover early precursors.<n>Our models support both continuous and discrete multivariate features, enabling robust performance in real-world settings.
  arXiv  Detail & Related papers  (2026-02-12T03:57:41Z)
• Keep the Lights On, Keep the Lengths in Check: Plug-In Adversarial Detection for Time-Series LLMs in Energy Forecasting [35.31487571405278]
  We propose a plug-in detection framework that capitalizes on the TS-LLM's own variable-length input capability.<n>We evaluate our approach on three representative TS-LLMs across three energy datasets.
  arXiv  Detail & Related papers  (2025-12-13T03:24:36Z)
• EMForecaster: A Deep Learning Framework for Time Series Forecasting in Wireless Networks with Distribution-Free Uncertainty Quantification [11.88654374102813]
  We develop a deep learning (DL) time series forecasting framework referred to as textitEMForecaster.<n> EMForecaster achieves superior performance across diverse datasets, spanning both short-term and long-term prediction horizons.<n>In point forecasting tasks, EMForecaster substantially outperforms current state-of-the-art DL approaches, showing improvements of 53.97% over the Transformer architecture and 38.44% over the average of all baseline models.
  arXiv  Detail & Related papers  (2025-03-31T18:10:08Z)
• Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving [31.995016095663544]
  LiDAR-generated occupancy grid maps (L-OGMs) offer a robust bird's-eye view for the scene representation.<n>Our proposed framework, Latent Occupancy Prediction (LOPR), performs L-OGM prediction in the latent space of a generative architecture.
  arXiv  Detail & Related papers  (2024-07-30T18:37:59Z)
• ARFA: An Asymmetric Receptive Field Autoencoder Model for Spatiotemporal Prediction [55.30913411696375]
  We propose an Asymmetric Receptive Field Autoencoder (ARFA) model, which introduces corresponding sizes of receptive field modules. In the encoder, we present large kernel module for globaltemporal feature extraction. In the decoder, we develop a small kernel module for localtemporal reconstruction. We construct the RainBench, a large-scale radar echo dataset for precipitation prediction, to address the scarcity of meteorological data in the domain.
  arXiv  Detail & Related papers  (2023-09-01T07:55:53Z)
• RARE: Robust Masked Graph Autoencoder [45.485891794905946]
  Masked graph autoencoder (MGAE) has emerged as a promising self-supervised graph pre-training (SGP) paradigm. We propose a novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the certainty in inferring masked data.
  arXiv  Detail & Related papers  (2023-04-04T03:35:29Z)
• Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System [46.39662315849883]
  We introduce CLMFormer, a novel framework that mitigates redundancy through curriculum learning and a memory-driven decoder.<n>CLMFormer consistently improves Transformer-based models by up to 30%, demonstrating its effectiveness in long-horizon forecasting.
  arXiv  Detail & Related papers  (2022-07-16T04:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.