Energy-Arena: A Dynamic Benchmark for Operational Energy Forecasting

• URL: http://arxiv.org/abs/2604.24705v1
• Date: Mon, 27 Apr 2026 17:14:01 GMT
• Title: Energy-Arena: A Dynamic Benchmark for Operational Energy Forecasting
• Abstract summary: This paper introduces the Energy-Arena, a dynamic benchmarking platform for operational energy time series forecasting.<n>By moving from retrospective backtesting to forward-looking benchmarking, the Energy-Arena enforces standardized ex-ante submission and ex-post evaluation.
• Score: 1.7225327025457389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Energy forecasting research faces a persistent comparability gap that makes it difficult to measure consistent progress over time. Reported accuracy gains are often not directly comparable because models are evaluated under study-specific datasets, time periods, information sets, and scoring setups, while widely used benchmarks and competition datasets are typically tied to fixed historical windows. This paper introduces the Energy-Arena, a dynamic benchmarking platform for operational energy time series forecasting that provides a continuously updated reference point as energy systems evolve. The platform operates as an open, API-based submission system and standardizes challenge definitions and submission deadlines aligned with operational constraints. Performance is reported on rolling evaluation windows via persistent leaderboards. By moving from retrospective backtesting to forward-looking benchmarking, the Energy-Arena enforces standardized ex-ante submission and ex-post evaluation, thereby improving transparency by preventing information leakage and retroactive tuning. The platform is publicly available at Energy-Arena.org.

Related papers

• LiveHouse-TS: An Open-world Living Benchmark for Time Series Foundation Models [41.28223590438589]
  We introduce LiveHouse-TS, the first open-world living benchmark infrastructure for Time Series Foundation Models.<n>LiveHouse-TS shifts time series benchmarking from snapshot accuracy to continuous temporal validity.<n>Our infrastructure serves as a continuous time series infrastructure designed to explore vital, long-term scientific questions.
  arXiv  Detail & Related papers  (2026-08-18T02:50:48Z)
• RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges [31.389267895745252]
  We introduce RealBench, a next-generation benchmark for AI weather forecasting that emphasizes realistic evaluation under operational conditions.<n>RealBench features a strictly out-of-distribution test set spanning 2025 to eliminate data leakage and capture recent atmospheric regimes.<n>It integrates multiple data sources, including low-latency operational analysis and a large-scale global in-situ observation dataset comprising over 10,000 stations.
  arXiv  Detail & Related papers  (2026-05-24T08:46:17Z)
• FRESCO: Benchmarking and Optimizing Re-rankers for Evolving Semantic Conflict in Retrieval-Augmented Generation [73.22935457705057]
  We introduce FRESCO, a benchmark for evaluating re-rankers in temporally dynamic contexts.<n>By pairing recency-seeking queries with historical Wikipedia revisions, FRESCO tests whether re-rankers can prioritize factually recent evidence while maintaining semantic relevance.<n>Our evaluation reveals a consistent failure mode across existing re-rankers: a strong bias toward older, semantically rich documents, even when they are factually obsolete.
  arXiv  Detail & Related papers  (2026-04-14T17:04:25Z)
• Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting [8.050703449557028]
  Impermanent is a live benchmark that evaluates forecasting models under open-world temporal change.<n>We focus on the top 400 repositories by star count and construct time series from issues opened, pull requests opened, push events, and new stargazers.<n>By shifting evaluation from static accuracy to sustained performance, Impermanent takes a concrete step toward assessing when and whether foundation-level generalization in time-series forecasting can be meaningfully claimed.
  arXiv  Detail & Related papers  (2026-03-09T17:59:00Z)
• It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks [87.7937890373758]
  Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.<n>We introduce TIME, a next-generation task-centric benchmark comprising 50 fresh datasets and 98 forecasting tasks.<n>We propose a novel pattern-level evaluation perspective that moves beyond traditional dataset-level evaluations based on static meta labels.
  arXiv  Detail & Related papers  (2026-02-12T16:31:01Z)
• TS-Arena Technical Report -- A Pre-registered Live Forecasting Platform [0.0]
  Time Series Foundation Models (TSFMs) offer transformative capabilities for forecasting, but risk triggering a fundamental evaluation crisis.<n>This crisis is driven by information leakage due to overlapping training and test sets across different models, as well as the illegitimate transfer of global patterns to test data.<n>We introduce TS-Arena, a platform that restores the operational integrity of forecasting by treating the genuinely unknown future as the definitive test environment.
  arXiv  Detail & Related papers  (2025-12-23T20:48:11Z)
• Evaluating End-User Device Energy Models in Sustainability Reporting of Browser-Based Web Services [0.0]
  Sustainability reporting in web-based services increasingly relies on simplified energy and carbon models.<n>This paper presents an empirical study evaluating how well such models reflect actual energy consumption.<n>Results show that the commonly applied constant-power approximation can diverge substantially from measured energy.
  arXiv  Detail & Related papers  (2025-10-14T14:25:26Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay.<n>Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks.<n>Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• OrionBench: Benchmarking Time Series Generative Models in the Service of the End-User [8.05635934199494]
  OrionBench is a continuous benchmarking framework for unsupervised time series anomaly detection models. We show how to use OrionBench, and the performance of pipelines across 17 releases published over the course of four years.
  arXiv  Detail & Related papers  (2023-10-26T19:43:16Z)
• Benchmarks and Custom Package for Energy Forecasting [55.460452605056894]
  Energy forecasting aims to minimize the cost of subsequent tasks such as power grid dispatch. In this paper, we collected large-scale load datasets and released a new renewable energy dataset. We conducted extensive experiments with 21 forecasting methods in these energy datasets at different levels under 11 evaluation metrics.
  arXiv  Detail & Related papers  (2023-07-14T06:50:02Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.