How to predict and optimise with asymmetric error metrics
- URL: http://arxiv.org/abs/2211.13586v1
- Date: Thu, 24 Nov 2022 13:16:45 GMT
- Title: How to predict and optimise with asymmetric error metrics
- Authors: Mahdi Abolghasemi, Richard Bean
- Abstract summary: We examine the concept of the predict and optimise problem with specific reference to the third Technical Challenge of the IEEE Computational Intelligence Society.
In this competition, entrants were asked to forecast building energy use and solar generation at six buildings and six solar installations, and then use their forecast to optimize energy cost while scheduling classes and batteries over a month.
We explore the different nature of loss functions for the prediction and optimisation phase and propose to adjust the final forecasts for a better optimisation cost.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we examine the concept of the predict and optimise problem
with specific reference to the third Technical Challenge of the IEEE
Computational Intelligence Society. In this competition, entrants were asked to
forecast building energy use and solar generation at six buildings and six
solar installations, and then use their forecast to optimize energy cost while
scheduling classes and batteries over a month. We examine the possible effect
of underforecasting and overforecasting and asymmetric errors on the
optimisation cost. We explore the different nature of loss functions for the
prediction and optimisation phase and propose to adjust the final forecasts for
a better optimisation cost. We report that while there is a positive
correlation between these two, more appropriate loss functions can be used to
optimise the costs associated with final decisions.
Related papers
- Predict. Optimize. Revise. On Forecast and Policy Stability in Energy Management Systems [3.3295510777293837]
This research addresses the challenge of integrating forecasting and optimization in energy management systems.
It proposes a novel framework for analyzing online optimization problems with switching costs and enabled by deterministic and probabilistic forecasts.
arXiv Detail & Related papers (2024-06-29T12:28:30Z) - Digital Twins for forecasting and decision optimisation with machine learning: applications in wastewater treatment [0.0]
Prediction and optimisation are techniques that have found many applications in solving real-world problems.
We review a digital twin that was developed and applied in wastewater treatment in Urban Utility to improve their operational efficiency.
arXiv Detail & Related papers (2024-04-23T00:18:20Z) - End-to-End Learning for Fair Multiobjective Optimization Under
Uncertainty [55.04219793298687]
The Predict-Then-Forecast (PtO) paradigm in machine learning aims to maximize downstream decision quality.
This paper extends the PtO methodology to optimization problems with nondifferentiable Ordered Weighted Averaging (OWA) objectives.
It shows how optimization of OWA functions can be effectively integrated with parametric prediction for fair and robust optimization under uncertainty.
arXiv Detail & Related papers (2024-02-12T16:33:35Z) - ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast.
We also introduce a training-free extreme value enhancement strategy named ExEnsemble, which increases the variance of pixel values and improves the forecast robustness.
Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
arXiv Detail & Related papers (2024-02-02T10:34:13Z) - Rethinking and Benchmarking Predict-then-Optimize Paradigm for
Combinatorial Optimization Problems [62.25108152764568]
Many web applications rely on solving optimization problems, such as energy cost-aware scheduling, budget allocation on web advertising, and graph matching on social networks.
We consider the performance of prediction and decision-making in a unified system.
We provide a comprehensive categorization of current approaches and integrate existing experimental scenarios.
arXiv Detail & Related papers (2023-11-13T13:19:34Z) - Prediction-Oriented Bayesian Active Learning [51.426960808684655]
Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters.
EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
arXiv Detail & Related papers (2023-04-17T10:59:57Z) - Comparison and Evaluation of Methods for a Predict+Optimize Problem in
Renewable Energy [42.00952788334554]
This paper presents the findings of the IEEE-CIS Technical Challenge on Predict+ for Renewable Energy Scheduling," held in 2021.
We present a comparison and evaluation of the seven highest-ranked solutions in the competition.
The winning method predicted different scenarios and optimized over all scenarios using a sample average approximation method.
arXiv Detail & Related papers (2022-12-21T02:34:12Z) - Multistep Multiappliance Load Prediction [0.0]
We develop a robust and accurate model for the appliance-level load prediction based on four datasets from four different regions.
The empirical results show that cyclical encoding of time features and weather indicators alongside a long-short term memory (LSTM) model offer the optimal performance.
arXiv Detail & Related papers (2022-12-19T13:01:51Z) - The Perils of Learning Before Optimizing [16.97597806975415]
We show how prediction models can be learned end-to-end by differentiating through the optimization task.
We show that the performance gap between a two-stage and end-to-end approach is closely related to the emphprice of correlation concept in optimization.
arXiv Detail & Related papers (2021-06-18T20:43:47Z) - Regret-Optimal Filtering [57.51328978669528]
We consider the problem of filtering in linear state-space models through the lens of regret optimization.
We formulate a novel criterion for filter design based on the concept of regret between the estimation error energy of a clairvoyant estimator.
We show that the regret-optimal estimator can be easily implemented by solving three Riccati equations and a single Lyapunov equation.
arXiv Detail & Related papers (2021-01-25T19:06:52Z) - Right Decisions from Wrong Predictions: A Mechanism Design Alternative
to Individual Calibration [107.15813002403905]
Decision makers often need to rely on imperfect probabilistic forecasts.
We propose a compensation mechanism ensuring that the forecasted utility matches the actually accrued utility.
We demonstrate an application showing how passengers could confidently optimize individual travel plans based on flight delay probabilities.
arXiv Detail & Related papers (2020-11-15T08:22:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.