How Understanding Forecast Uncertainty Resolves the Explainability Problem in Machine Learning Models

• URL: http://arxiv.org/abs/2602.00179v1
• Date: Fri, 30 Jan 2026 04:43:06 GMT
• Title: How Understanding Forecast Uncertainty Resolves the Explainability Problem in Machine Learning Models
• Authors: Joseph L. Breeden,
• Abstract summary: Local linear methods for generating explanations have been criticized for being unstable near decision boundaries.<n>We show that such concerns reflect a misunderstanding of the problem.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: For applications of machine learning in critical decisions, explainability is a primary concern, and often a regulatory requirement. Local linear methods for generating explanations, such as LIME and SHAP, have been criticized for being unstable near decision boundaries. In this paper, we explain that such concerns reflect a misunderstanding of the problem. The forecast uncertainty is high at decision boundaries, so consequently, the explanatory instability is high. The correct approach is to change the sequence of events and questions being asked. Nonlinear models can be highly predictive in some regions while having little or no predictability in others. Therefore, the first question is whether a usable forecast exists. When there is a forecast with low enough uncertainty to be useful, an explanation can be sought via a local linear approximation. In such cases, the explanatory instability is correspondingly low. When no usable forecast exists, the decision must fall to a simpler overall model such as traditional logistic regression. Additionally, these results show that some methods that purport to be explainable everywhere, such as ReLU networks or any piecewise linear model, have only an illusory explainability, because the forecast uncertainty at the segment boundaries is too high to be useful. Explaining an unusable forecast is pointless.

Related papers

• XForecast: Evaluating Natural Language Explanations for Time Series Forecasting [72.57427992446698]
  Time series forecasting aids decision-making, especially for stakeholders who rely on accurate predictions. Traditional explainable AI (XAI) methods, which underline feature or temporal importance, often require expert knowledge. evaluating forecast NLEs is difficult due to the complex causal relationships in time series data.
  arXiv  Detail & Related papers  (2024-10-18T05:16:39Z)
• Identifying Drivers of Predictive Aleatoric Uncertainty [2.5311562666866494]
  We propose a straightforward approach to explain predictive aleatoric uncertainties.<n>We estimate uncertainty in regression as predictive variance by adapting a neural network with a Gaussian output distribution.<n>This approach can explain uncertainty influences more reliably than complex published approaches.
  arXiv  Detail & Related papers  (2023-12-12T13:28:53Z)
• Calibrated Explanations for Regression [1.2058600649065616]
  Calibrated Explanations for regression provides fast, reliable, stable, and robust explanations. Calibrated Explanations for probabilistic regression provides an entirely new way of creating explanations. An implementation in Python is freely available on GitHub and for installation using both pip and conda.
  arXiv  Detail & Related papers  (2023-08-30T18:06:57Z)
• Model Agnostic Local Explanations of Reject [6.883906273999368]
  The application of machine learning based decision making systems in safety critical areas requires reliable high certainty predictions. Reject options are a common way of ensuring a sufficiently high certainty of predictions made by the system. We propose a model agnostic method for locally explaining arbitrary reject options by means of interpretable models and counterfactual explanations.
  arXiv  Detail & Related papers  (2022-05-16T12:42:34Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Robust uncertainty estimates with out-of-distribution pseudo-inputs training [0.0]
  We propose to explicitly train the uncertainty predictor where we are not given data to make it reliable. As one cannot train without data, we provide mechanisms for generating pseudo-inputs in informative low-density regions of the input space. With a holistic evaluation, we demonstrate that this yields robust and interpretable predictions of uncertainty while retaining state-of-the-art performance on diverse tasks.
  arXiv  Detail & Related papers  (2022-01-15T17:15:07Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Dense Uncertainty Estimation [62.23555922631451]
  In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation. We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
  arXiv  Detail & Related papers  (2021-10-13T01:23:48Z)
• Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression [91.3373131262391]
  Uncertainty is the only certainty there is. Traditionally, the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. How to model the uncertainty within the present-day technologies for regression remains an open issue.
  arXiv  Detail & Related papers  (2021-03-25T06:56:09Z)
• Learnable Uncertainty under Laplace Approximations [65.24701908364383]
  We develop a formalism to explicitly "train" the uncertainty in a decoupled way to the prediction itself. We show that such units can be trained via an uncertainty-aware objective, improving standard Laplace approximations' performance.
  arXiv  Detail & Related papers  (2020-10-06T13:43:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.