Distributed JEPA: A Self-Supervised Framework for Energy Forecasting

• URL: http://arxiv.org/abs/2609.17029v1
• Date: Tue, 15 Sep 2026 11:36:20 GMT
• Title: Distributed JEPA: A Self-Supervised Framework for Energy Forecasting
• Abstract summary: We propose a distributed Joint Embedding Predictive Architecture (JEPA) for self-supervised learning from heterogeneous energy time-series.<n>The framework predicts latent representations of masked temporal segments while integrating temporal observations and contextual information within a shared embedding space.<n>JEPA achieved performance comparable to a Transformer on building energy data, higher $R2$ in 3/5 consumer clusters, and outperformed the baseline on 9/10 unseen PVs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traditional energy forecasting solutions rely on task-specific supervision and energy asset representations, limiting transferability and the ability to capture general temporal dynamics across heterogeneous assets. We address this by proposing a distributed Joint Embedding Predictive Architecture (JEPA) for self-supervised learning from heterogeneous energy time-series. The framework predicts latent representations of masked temporal segments while integrating temporal observations and contextual information within a shared embedding space. To prevent representation collapse, training combines a latent-space predictive objective with covariance and temporal variance regularization. The evaluation was conducted on energy consumption and generation datasets under data-degradation scenarios and compared with a Transformer forecasting baseline. The learned representations remained stable (cosine similarity $\approx 0.98$; effective rank 185-235). JEPA achieved performance comparable to a Transformer on building energy data, higher $R^2$ in 3/5 consumer clusters, and outperformed the baseline on 9/10 unseen PVs ($R^2$=0.73-0.88 vs. <0.45), while showing greater robustness to missing data.

Related papers

• Improving Coherence in Hierarchical Time Series Forecasting using Structured Temporal Fusion [0.0]
  Hierarchical Temporal Fusion (HTF) integrates structured hierarchical embeddings with a coherence-aware loss function to ensure consistent forecasts across all levels of a hierarchy.<n>We evaluate HTF on two publicly available benchmark datasets: the M5 Walmart forecasting dataset and a publicly available hierarchical energy consumption dataset.
  arXiv  Detail & Related papers  (2026-06-26T19:11:40Z)
• Spatio-temporal modelling of electric vehicle charging demand [0.41998444721319217]
  We introduce a novel-scale longitudinal dataset collected across Scotland (2022 2025)<n>We formulate EV charging demand as a latent latent latent Gaussian field perform approximate inference via Integrated Nested Laplace Approximation (INLA)<n>On station-level forecasting tasks, our approach achieves competitive predictive accuracy against machine learning baselines, while additionally providing principled uncertainty quan tification and interpretable spatial and temporal decompositions properties that are essential for risk-aware infrastructure planning.
  arXiv  Detail & Related papers  (2026-04-21T09:52:22Z)
• $V_{0.5}$: Generalist Value Model as a Prior for Sparse RL Rollouts [81.48669089692189]
  Generalist Value Models (such as $V_0.5$) achieve pre-trained value estimation by explicitly encoding model capabilities in-context.<n>We propose $V_0.5$, which adaptively fuses the baseline predicted by such value model with the empirical mean derived from sparse rollouts.<n>$V_0.5$ significantly outperforms GRPO and DAPO, achieving faster convergence and over some 10% performance improvement.
  arXiv  Detail & Related papers  (2026-03-11T14:57:41Z)
• The Living Forecast: Evolving Day-Ahead Predictions into Intraday Reality [0.688204255655161]
  This paper introduces a Bayesian updating mechanism that converts fully probabilistic day-ahead forecasts into intraday forecasts without retraining or re-inference.<n>Experiments on household electricity consumption and photovoltaic generation datasets demonstrate that the proposed method improves forecast accuracy up to 25% across likelihood-, sample-, quantile-, and point-based metrics.
  arXiv  Detail & Related papers  (2025-10-14T08:25:20Z)
• PowerGrow: Feasible Co-Growth of Structures and Dynamics for Power Grid Synthesis [75.14189839277928]
  We present PowerGrow, a co-generative framework that significantly reduces computational overhead while maintaining operational validity.<n> Experiments across benchmark settings show that PowerGrow outperforms prior diffusion models in fidelity and diversity.<n>This demonstrates its ability to generate operationally valid and realistic power grid scenarios.
  arXiv  Detail & Related papers  (2025-08-29T01:47:27Z)
• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay.<n>Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks.<n>Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• Weather-Informed Probabilistic Forecasting and Scenario Generation in Power Systems [15.393565192962482]
  Integration of renewable energy sources into power grids presents significant challenges due to their intrinsicity and uncertainty. This paper proposes a method combining probabilistic forecasting and Gaussian copula for day-ahead prediction and scenario generation of wind, and solar power in high-dimensional contexts.
  arXiv  Detail & Related papers  (2024-09-11T21:44:59Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Benchmarks and Custom Package for Energy Forecasting [55.460452605056894]
  Energy forecasting aims to minimize the cost of subsequent tasks such as power grid dispatch. In this paper, we collected large-scale load datasets and released a new renewable energy dataset. We conducted extensive experiments with 21 forecasting methods in these energy datasets at different levels under 11 evaluation metrics.
  arXiv  Detail & Related papers  (2023-07-14T06:50:02Z)
• Long-term Wind Power Forecasting with Hierarchical Spatial-Temporal Transformer [112.12271800369741]
  Wind power is attracting increasing attention around the world due to its renewable, pollution-free, and other advantages. Accurate wind power forecasting (WPF) can effectively reduce power fluctuations in power system operations. Existing methods are mainly designed for short-term predictions and lack effective spatial-temporal feature augmentation.
  arXiv  Detail & Related papers  (2023-05-30T04:03:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.