RADMI: Latent Information Aggregation as a Proxy for Model Uncertainty

• URL: http://arxiv.org/abs/2605.01502v1
• Date: Sat, 02 May 2026 15:49:25 GMT
• Title: RADMI: Latent Information Aggregation as a Proxy for Model Uncertainty
• Abstract summary: We propose a single-pass method that estimates prediction uncertainty by measuring mutual information between decoder layers in segmentation networks.<n>RADMI achieves the highest correlation with deep ensemble uncertainty among all single-pass methods.<n>Our results suggest that linear aggregation of normalized information flow provides a principled and efficient proxy for prediction uncertainty in encoder-decoder architectures.
• Score: 12.261505821589452
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Epistemic uncertainty estimation is essential for identifying regions where deep learning system outputs may be unreliable. However, existing approaches require computationally expensive ensemble methods or multiple stochastic forward passes, limiting their scalability to dense prediction tasks like segmentation. We propose Resolution-Aggregated Decoder Mutual Information (RADMI), a single-pass method that estimates prediction uncertainty by measuring mutual information (MI) between consecutive decoder layers in segmentation networks. We observe that elevated inter-layer MI correlates with prediction uncertainty, as the network must integrate conflicting contextual information at ambiguous regions such as class boundaries. Evaluating on a seismic facies segmentation benchmark, RADMI achieves the highest correlation with deep ensemble uncertainty among all single-pass methods, outperforming the next-best baselines by 5.5% in Pearson and 10.7% in Spearman correlation coefficients. Compared to baselines that either lack spatial precision or demand significant computational overhead, RADMI yields sharp, boundary-localized uncertainty maps without architectural modifications. Our results suggest that linear aggregation of normalized information flow provides a principled and efficient proxy for prediction uncertainty in encoder-decoder architectures.

Related papers

• Probabilistic indirect models for undrained shear strength: addressing significant data missing and variability with advanced imputation and machine learning techniques [3.6913038494792656]
  Accurate prediction of undrained shear strength (su) is crucial for geotechnical design, but is often hampered by substantial uncertainty.<n>This study uses the CLAY/10/7490 global database to develop probabilistic indirect models to predict su.
  arXiv  Detail & Related papers  (2026-08-14T04:15:16Z)
• Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation [9.817102014355617]
  We show that well-calibrated predictive uncertainty estimates can be extracted from a mixture of experts (MoE) without architectural modifications.<n>Our results show that MoEs yield more reliable uncertainty estimates than ensembles in terms of conditional correctness metrics.<n>Our experiments on the Cityscapes dataset suggest that increasing the number of experts can further enhance uncertainty calibration.
  arXiv  Detail & Related papers  (2025-09-05T05:30:53Z)
• Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data [49.36938105983916]
  Unsupervised domain adaptation methods seek to generalize effectively on unlabeled test data.<n>We propose incorporating multi-scale feature extraction and uncertainty estimation to improve the model's generalization and robustness across domains.
  arXiv  Detail & Related papers  (2025-08-26T03:13:08Z)
• Conformal Information Pursuit for Interactively Guiding Large Language Models [68.16703423481935]
  This paper explores sequential querying strategies that aim to minimize the expected number of queries.<n>One such strategy is Information Pursuit (IP), a greedy algorithm that at each iteration selects the query that maximizes information gain or equivalently minimizes uncertainty.<n>We propose Conformal Information Pursuit (C-IP), an alternative approach to sequential information gain based on conformal prediction sets.
  arXiv  Detail & Related papers  (2025-07-04T03:55:39Z)
• A Deep Bayesian Nonparametric Framework for Robust Mutual Information Estimation [9.68824512279232]
  Mutual Information (MI) is a crucial measure for capturing dependencies between variables.<n>We present a solution for training an MI estimator by constructing the MI loss with a finite representation of the Dirichlet process posterior to incorporate regularization.<n>We explore the application of our estimator in maximizing MI between the data space and the latent space of a variational autoencoder.
  arXiv  Detail & Related papers  (2025-03-11T21:27:48Z)
• Information Leakage Detection through Approximate Bayes-optimal Prediction [22.04308347355652]
  Information leakage (IL) involves unintentionally exposing sensitive information to unauthorized parties.<n> Conventional statistical approaches rely on estimating mutual information between observable and secret information for detecting ILs.<n>We establish a theoretical framework using statistical learning theory and information theory to quantify and detect IL accurately.
  arXiv  Detail & Related papers  (2024-01-25T16:15:27Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• Uncertainty Estimation for Heatmap-based Landmark Localization [4.673063715963989]
  We propose Quantile Binning, a data-driven method to categorise predictions by uncertainty with estimated error bounds. We demonstrate this framework by comparing and contrasting three uncertainty measures. We conclude by illustrating how filtering out gross mispredictions caught in our Quantile Bins significantly improves the proportion of predictions under an acceptable error threshold.
  arXiv  Detail & Related papers  (2022-03-04T14:40:44Z)
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty [58.144520501201995]
  Bi-Lipschitz regularization of neural network layers preserve relative distances between data instances in the feature spaces of each layer. With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices. We also propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution.
  arXiv  Detail & Related papers  (2021-10-12T22:04:19Z)
• Gradient-Based Quantification of Epistemic Uncertainty for Deep Object Detectors [8.029049649310213]
  We introduce novel gradient-based uncertainty metrics and investigate them for different object detection architectures. Experiments show significant improvements in true positive / false positive discrimination and prediction of intersection over union. We also find improvement over Monte-Carlo dropout uncertainty metrics and further significant boosts by aggregating different sources of uncertainty metrics.
  arXiv  Detail & Related papers  (2021-07-09T16:04:11Z)
• Amortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation [99.92568326314667]
  We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation. Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle. We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-11-05T08:04:34Z)
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
  We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
  arXiv  Detail & Related papers  (2020-03-04T12:27:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.