Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification

• URL: http://arxiv.org/abs/2511.10282v1
• Date: Fri, 14 Nov 2025 01:43:32 GMT
• Title: Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification
• Authors: Adrien Lafage, Olivier Laurent, Firas Gabetni, Gianni Franchi,
• Abstract summary: Uncertainty Quantification (UQ) for Deep Learning aims to improve the reliability of uncertainty estimates.<n>We introduce Torch-Uncertainty, a PyTorch and Lightning-based framework designed to streamline training and evaluation.<n>We present comprehensive experimental results that benchmark a diverse set of UQ methods across classification, segmentation, and regression tasks.
• Score: 11.898587151486709
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Neural Networks (DNNs) have demonstrated remarkable performance across various domains, including computer vision and natural language processing. However, they often struggle to accurately quantify the uncertainty of their predictions, limiting their broader adoption in critical real-world applications. Uncertainty Quantification (UQ) for Deep Learning seeks to address this challenge by providing methods to improve the reliability of uncertainty estimates. Although numerous techniques have been proposed, a unified tool offering a seamless workflow to evaluate and integrate these methods remains lacking. To bridge this gap, we introduce Torch-Uncertainty, a PyTorch and Lightning-based framework designed to streamline DNN training and evaluation with UQ techniques and metrics. In this paper, we outline the foundational principles of our library and present comprehensive experimental results that benchmark a diverse set of UQ methods across classification, segmentation, and regression tasks. Our library is available at https://github.com/ENSTA-U2IS-AI/Torch-Uncertainty

Related papers

• UncertaintyZoo: A Unified Toolkit for Quantifying Predictive Uncertainty in Deep Learning Systems [5.790749437470997]
  Large language models (LLMs) are increasingly expanding their real-world applications across domains.<n>Despite this achievement, LLMs often make incorrect predictions, which can lead to potential losses in safety-critical scenarios.<n>We introduce UncertaintyZoo, a unified toolkit that integrates 29 uncertainty quantification methods.
  arXiv  Detail & Related papers  (2025-12-06T11:45:50Z)
• UNCERTAINTY-LINE: Length-Invariant Estimation of Uncertainty for Large Language Models [51.53270695871237]
  We show that UNCERTAINTY-LINE: consistently improves over even nominally length-normalized UQ methods uncertainty estimates.<n>Our method is post-hoc, model-agnostic, and applicable to a range of UQ measures.
  arXiv  Detail & Related papers  (2025-05-25T09:30:43Z)
• Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models [76.17975723711886]
  Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs)<n>In this work, we adapt Mahalanobis Distance (MD) - a well-established UQ technique in classification tasks - for text generation.<n>Our method extracts token embeddings from multiple layers of LLMs, computes MD scores for each token, and uses linear regression trained on these features to provide robust uncertainty scores.
  arXiv  Detail & Related papers  (2025-02-20T10:25:13Z)
• Uncertainty Quantification with the Empirical Neural Tangent Kernel [13.132499571608868]
  We propose a post-hoc, sampling-based UQ method for over- parameterized networks at the end of training.<n>Our approach constructs efficient and meaningful deep ensembles by employing a (stochastic) gradient-descent sampling process.<n>We show that our method not only outperforms competing approaches in computational efficiency-often reducing costs by multiple factors-but also maintains state-of-the-art performance across a variety of UQ metrics for both regression and classification tasks.
  arXiv  Detail & Related papers  (2025-02-05T04:01:34Z)
• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [83.90988015005934]
  Uncertainty quantification is a key element of machine learning applications.<n>We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines.<n>We conduct a large-scale empirical investigation of UQ and normalization techniques across eleven tasks, identifying the most effective approaches.
  arXiv  Detail & Related papers  (2024-06-21T20:06:31Z)
• Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution [110.99891169486366]
  We propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions.
  arXiv  Detail & Related papers  (2024-02-13T11:22:59Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• A Survey on Uncertainty Quantification Methods for Deep Learning [7.102893202197349]
  Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy.<n>This paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources.<n>We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems.
  arXiv  Detail & Related papers  (2023-02-26T22:30:08Z)
• Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning [66.59455427102152]
  We introduce Uncertainty Baselines: high-quality implementations of standard and state-of-the-art deep learning methods on a variety of tasks. Each baseline is a self-contained experiment pipeline with easily reusable and extendable components. We provide model checkpoints, experiment outputs as Python notebooks, and leaderboards for comparing results.
  arXiv  Detail & Related papers  (2021-06-07T23:57:32Z)
• Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification [4.56877715768796]
  Uncertainty-wizard is a tool that allows to quantify such uncertainty and confidence in artificial neural networks. It is built on top of the industry-leading tf.keras deep learning API and it provides a near-transparent and easy to understand interface.
  arXiv  Detail & Related papers  (2020-12-29T15:38:24Z)
• Fast Uncertainty Quantification for Deep Object Pose Estimation [91.09217713805337]
  Deep learning-based object pose estimators are often unreliable and overconfident. In this work, we propose a simple, efficient, and plug-and-play UQ method for 6-DoF object pose estimation.
  arXiv  Detail & Related papers  (2020-11-16T06:51:55Z)
• Confidence-Aware Learning for Deep Neural Networks [4.9812879456945]
  We propose a method of training deep neural networks with a novel loss function, named Correctness Ranking Loss. It regularizes class probabilities explicitly to be better confidence estimates in terms of ordinal ranking according to confidence. It has almost the same computational costs for training as conventional deep classifiers and outputs reliable predictions by a single inference.
  arXiv  Detail & Related papers  (2020-07-03T02:00:35Z)
• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning [70.72363097550483]
  In this study, we focus on in-domain uncertainty for image classification. To provide more insight in this study, we introduce the deep ensemble equivalent score (DEE)
  arXiv  Detail & Related papers  (2020-02-15T23:28:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.