Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models

• URL: http://arxiv.org/abs/2407.21299v1
• Date: Wed, 31 Jul 2024 02:57:21 GMT
• Title: Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models
• Authors: Kaustav Bhattacharjee, Soumya Kundu, Indrasis Chakraborty, Aritra Dasgupta,
• Abstract summary: This paper introduces a visual analytics-based application designed to compare the performance of deep-learning-based net load forecasting models with other models for probabilistic net load forecasting. The application employs carefully selected visual analytic interventions, enabling users to discern differences in model performance across different solar penetration levels, dataset resolutions, and hours of the day over multiple months.
• Score: 0.562479170374811
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Net load forecasting is crucial for energy planning and facilitating informed decision-making regarding trade and load distributions. However, evaluating forecasting models' performance against benchmark models remains challenging, thereby impeding experts' trust in the model's performance. In this context, there is a demand for technological interventions that allow scientists to compare models across various timeframes and solar penetration levels. This paper introduces a visual analytics-based application designed to compare the performance of deep-learning-based net load forecasting models with other models for probabilistic net load forecasting. This application employs carefully selected visual analytic interventions, enabling users to discern differences in model performance across different solar penetration levels, dataset resolutions, and hours of the day over multiple months. We also present observations made using our application through a case study, demonstrating the effectiveness of visualizations in aiding scientists in making informed decisions and enhancing trust in net load forecasting models.

Related papers

• ModelRadar: Aspect-based Forecast Evaluation [0.393259574660092]
  Current practices for evaluating and comparing forecasting models focus on summarising performance into a single score. This limitation is especially problematic for time series forecasting, where multiple layers of averaging--across time steps, horizons, and multiple time series in a dataset--can mask relevant performance variations. We propose ModelRadar, a framework for evaluating univariate time series forecasting models across multiple aspects, such as stationarity, presence of anomalies, or forecasting horizons.
  arXiv  Detail & Related papers  (2025-03-31T11:50:45Z)
• Hourly Short Term Load Forecasting for Residential Buildings and Energy Communities [0.0]
  We introduce persistence models, auto-regressive-based machine learning models, and more advanced deep learning models. We observe a 15-30% increase in the prediction accuracy of the newly introduced hourly-based forecasting models over existing approaches.
  arXiv  Detail & Related papers  (2025-01-31T15:49:09Z)
• Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy [104.48511402784763]
  Performance Law for SR models aims to theoretically investigate and model the relationship between model performance and data quality. We propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics.
  arXiv  Detail & Related papers  (2024-11-30T10:56:30Z)
• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
  We propose a novel approach to explain the behavior of a black-box model under feature shifts. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
  arXiv  Detail & Related papers  (2024-08-24T18:28:19Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Weather Prediction with Diffusion Guided by Realistic Forecast Processes [49.07556359513563]
  We introduce a novel method that applies diffusion models (DM) for weather forecasting. Our method can achieve both direct and iterative forecasting with the same modeling framework. The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community.
  arXiv  Detail & Related papers  (2024-02-06T21:28:42Z)
• Forte: An Interactive Visual Analytic Tool for Trust-Augmented Net Load Forecasting [0.6144680854063939]
  We present Forte, a visual analytics-based application to explore deep probabilistic net load forecasting models across various input variables. We discuss observations made using Forte and demonstrate the effectiveness of visualization techniques to provide valuable insights into the correlation between weather inputs and net load forecasts.
  arXiv  Detail & Related papers  (2023-11-10T22:15:11Z)
• A comparative assessment of deep learning models for day-ahead load forecasting: Investigating key accuracy drivers [2.572906392867547]
  Short-term load forecasting (STLF) is vital for the effective and economic operation of power grids and energy markets. Several deep learning models have been proposed in the literature for STLF, reporting promising results.
  arXiv  Detail & Related papers  (2023-02-23T17:11:04Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)
• Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning [109.74041512359476]
  We study a number of design decisions for the predictive model in visual MBRL algorithms. We find that a range of design decisions that are often considered crucial, such as the use of latent spaces, have little effect on task performance. We show how this phenomenon is related to exploration and how some of the lower-scoring models on standard benchmarks will perform the same as the best-performing models when trained on the same training data.
  arXiv  Detail & Related papers  (2020-12-08T18:03:21Z)
• Introduction to Rare-Event Predictive Modeling for Inferential Statisticians -- A Hands-On Application in the Prediction of Breakthrough Patents [0.0]
  We introduce a machine learning (ML) approach to quantitative analysis geared towards optimizing the predictive performance. We discuss the potential synergies between the two fields against the backdrop of this, at first glance, target-incompatibility. We are providing a hands-on predictive modeling introduction for a quantitative social science audience while aiming at demystifying computer science jargon.
  arXiv  Detail & Related papers  (2020-03-30T13:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.