Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation

• URL: http://arxiv.org/abs/2602.08142v1
• Date: Sun, 08 Feb 2026 22:05:23 GMT
• Title: Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation
• Authors: H. Martin Gillis, Isaac Xu, Thomas Trappenberg,
• Abstract summary: Variance-Gated Ensembles (VGE) is an intuitive framework that injects epistemic sensitivity via a signal-to-noise gate computed from ensemble statistics.<n>We derive closed-form vector-Jacobian products enabling end-to-end training through ensemble sample mean and variance.
• Score: 0.6340400318304492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning applications require fast and reliable per-sample uncertainty estimation. A common approach is to use predictive distributions from Bayesian or approximation methods and additively decompose uncertainty into aleatoric (i.e., data-related) and epistemic (i.e., model-related) components. However, additive decomposition has recently been questioned, with evidence that it breaks down when using finite-ensemble sampling and/or mismatched predictive distributions. This paper introduces Variance-Gated Ensembles (VGE), an intuitive, differentiable framework that injects epistemic sensitivity via a signal-to-noise gate computed from ensemble statistics. VGE provides: (i) a Variance-Gated Margin Uncertainty (VGMU) score that couples decision margins with ensemble predictive variance; and (ii) a Variance-Gated Normalization (VGN) layer that generalizes the variance-gated uncertainty mechanism to training via per-class, learnable normalization of ensemble member probabilities. We derive closed-form vector-Jacobian products enabling end-to-end training through ensemble sample mean and variance. VGE matches or exceeds state-of-the-art information-theoretic baselines while remaining computationally efficient. As a result, VGE provides a practical and scalable approach to epistemic-aware uncertainty estimation in ensemble models. An open-source implementation is available at: https://github.com/nextdevai/vge.

Related papers

• In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning [51.56484100374058]
  We introduce a principled risk decomposition that separates the total ICL risk into two components: Bayes Gap and Posterior Variance.<n>For a uniform-attention Transformer, we derive a non-asymptotic upper bound on this gap, which explicitly clarifies the dependence on the number of pretraining prompts.<n>The Posterior Variance is a model-independent risk representing the intrinsic task uncertainty.
  arXiv  Detail & Related papers  (2025-10-13T03:42:31Z)
• Multidimensional Uncertainty Quantification via Optimal Transport [87.97146725546502]
  We take a multidimensional view on uncertainty quantification (UQ) by stacking complementary UQ measures into a vector.<n>VecUQ-OT shows high efficiency even when individual measures fail.
  arXiv  Detail & Related papers  (2025-09-26T14:09:03Z)
• Uncertainty Estimation using Variance-Gated Distributions [0.6340400318304492]
  We propose an intuitive framework for uncertainty estimation and decomposition based on the signal-to-noise ratio of class probability distributions.<n>We introduce a variance-gated measure that scales predictions by a confidence factor derived from ensembles.
  arXiv  Detail & Related papers  (2025-09-07T16:19:21Z)
• MVG-CRPS: A Robust Loss Function for Multivariate Probabilistic Forecasting [17.212396544233307]
  We propose MVG-CRPS, a strictly proper scoring rule for MVG distributions.<n>MVG-CRPS admits a closed-form expression in terms of neural network outputs, thereby integrating seamlessly into deep learning frameworks.
  arXiv  Detail & Related papers  (2024-10-11T15:10:20Z)
• Uncertainty Estimation and Out-of-Distribution Detection for LiDAR Scene Semantic Segmentation [0.6144680854063939]
  Safe navigation in new environments requires autonomous vehicles and robots to accurately interpret their surroundings. We propose a method to distinguish in-distribution (ID) from out-of-distribution (OOD) samples. We quantify both epistemic and aleatoric uncertainties using the feature space of a single deterministic model.
  arXiv  Detail & Related papers  (2024-10-11T10:19:24Z)
• Invariant Probabilistic Prediction [45.90606906307022]
  We show that arbitrary distribution shifts do not, in general, admit invariant and robust probabilistic predictions. We propose a method to yield invariant probabilistic predictions, called IPP, and study the consistency of the underlying parameters.
  arXiv  Detail & Related papers  (2023-09-18T18:50:24Z)
• Quantification of Predictive Uncertainty via Inference-Time Sampling [57.749601811982096]
  We propose a post-hoc sampling strategy for estimating predictive uncertainty accounting for data ambiguity. The method can generate different plausible outputs for a given input and does not assume parametric forms of predictive distributions.
  arXiv  Detail & Related papers  (2023-08-03T12:43:21Z)
• Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition [7.811916700683125]
  We introduce a bias-variance decomposition for proper scores, giving rise to the Bregman Information as the variance term. We showcase the practical relevance of this decomposition on several downstream tasks, including model ensembles and confidence regions.
  arXiv  Detail & Related papers  (2022-10-21T21:24:37Z)
• Predicting Out-of-Domain Generalization with Neighborhood Invariance [59.05399533508682]
  We propose a measure of a classifier's output invariance in a local transformation neighborhood. Our measure is simple to calculate, does not depend on the test point's true label, and can be applied even in out-of-domain (OOD) settings. In experiments on benchmarks in image classification, sentiment analysis, and natural language inference, we demonstrate a strong and robust correlation between our measure and actual OOD generalization.
  arXiv  Detail & Related papers  (2022-07-05T14:55:16Z)
• CovarianceNet: Conditional Generative Model for Correct Covariance Prediction in Human Motion Prediction [71.31516599226606]
  We present a new method to correctly predict the uncertainty associated with the predicted distribution of future trajectories. Our approach, CovariaceNet, is based on a Conditional Generative Model with Gaussian latent variables.
  arXiv  Detail & Related papers  (2021-09-07T09:38:24Z)
• Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
  We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
  arXiv  Detail & Related papers  (2021-06-07T17:44:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.