Evidential Deep Learning for Uncertainty Quantification and Out-of-Distribution Detection in Jet Identification using Deep Neural Networks

• URL: http://arxiv.org/abs/2501.05656v1
• Date: Fri, 10 Jan 2025 02:14:29 GMT
• Title: Evidential Deep Learning for Uncertainty Quantification and Out-of-Distribution Detection in Jet Identification using Deep Neural Networks
• Authors: Ayush Khot, Xiwei Wang, Avik Roy, Volodymyr Kindratenko, Mark S. Neubauer,
• Abstract summary: We use evidential deep learning (EDL) for deep neural network models designed to identify jets in proton-proton collisions at the Large Hadron Collider. EDL treats learning as an evidence acquisition process designed to provide confidence about test data. We show how EDL quantifies uncertainty and detects out-of-distribution data which may lead to improved EDL methods for DL models applied to classification tasks.
• Score: 0.6558603851407393
• License:
• Abstract: Current methods commonly used for uncertainty quantification (UQ) in deep learning (DL) models utilize Bayesian methods which are computationally expensive and time-consuming. In this paper, we provide a detailed study of UQ based on evidential deep learning (EDL) for deep neural network models designed to identify jets in high energy proton-proton collisions at the Large Hadron Collider and explore its utility in anomaly detection. EDL is a DL approach that treats learning as an evidence acquisition process designed to provide confidence (or epistemic uncertainty) about test data. Using publicly available datasets for jet classification benchmarking, we explore hyperparameter optimizations for EDL applied to the challenge of UQ for jet identification. We also investigate how the uncertainty is distributed for each jet class, how this method can be implemented for the detection of anomalies, how the uncertainty compares with Bayesian ensemble methods, and how the uncertainty maps onto latent spaces for the models. Our studies uncover some pitfalls of EDL applied to anomaly detection and a more effective way to quantify uncertainty from EDL as compared with the foundational EDL setup. These studies illustrate a methodological approach to interpreting EDL in jet classification models, providing new insights on how EDL quantifies uncertainty and detects out-of-distribution data which may lead to improved EDL methods for DL models applied to classification tasks.

Related papers

• Calibrating LLMs with Information-Theoretic Evidential Deep Learning [22.263501423275784]
  Fine-tuned large language models (LLMs) often exhibit overconfidence, particularly when trained on small datasets. Evidential Deep Learning (EDL), an uncertainty-aware approach, enables uncertainty estimation in a single forward pass. We propose regularizing EDL by incorporating an information bottleneck (IB)
  arXiv  Detail & Related papers  (2025-02-10T11:00:24Z)
• Generative Edge Detection with Stable Diffusion [52.870631376660924]
  Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. We propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. We conduct extensive experiments on multiple datasets and achieve competitive performance.
  arXiv  Detail & Related papers  (2024-10-04T01:52:23Z)
• Uncertainty Estimation by Density Aware Evidential Deep Learning [7.328039160501826]
  Evidential deep learning (EDL) has shown remarkable success in uncertainty estimation. We propose a novel method called Density Aware Evidential Deep Learning (DAEDL) DAEDL integrates the feature space density of the testing example with the output of EDL during the prediction stage. It demonstrates state-of-the-art performance across diverse downstream tasks related to uncertainty estimation and classification.
  arXiv  Detail & Related papers  (2024-09-13T12:04:45Z)
• A Comprehensive Survey on Evidential Deep Learning and Its Applications [64.83473301188138]
  Evidential Deep Learning (EDL) provides reliable uncertainty estimation with minimal additional computation in a single forward pass. We first delve into the theoretical foundation of EDL, the subjective logic theory, and discuss its distinctions from other uncertainty estimation frameworks. We elaborate on its extensive applications across various machine learning paradigms and downstream tasks.
  arXiv  Detail & Related papers  (2024-09-07T05:55:06Z)
• Anomaly Detection of Tabular Data Using LLMs [54.470648484612866]
  We show that pre-trained large language models (LLMs) are zero-shot batch-level anomaly detectors. We propose an end-to-end fine-tuning strategy to bring out the potential of LLMs in detecting real anomalies.
  arXiv  Detail & Related papers  (2024-06-24T04:17:03Z)
• Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? [35.15844215216846]
  EDL methods are trained to learn a meta distribution over the predictive distribution by minimizing a specific objective function. Recent studies identify limitations of the existing methods to conclude their learned uncertainties are unreliable. We provide a sharper understanding of the behavior of a wide class of EDL methods by unifying various objective functions. We conclude that even when EDL methods are empirically effective on downstream tasks, this occurs despite their poor uncertainty quantification capabilities.
  arXiv  Detail & Related papers  (2024-02-09T03:23:39Z)
• Knowledge from Uncertainty in Evidential Deep Learning [10.751990848772028]
  This work reveals an evidential signal that emerges from the uncertainty value in Evidential Deep Learning (EDL) EDL is one example of a class of uncertainty-aware deep learning approaches designed to provide confidence about the current test sample.
  arXiv  Detail & Related papers  (2023-10-19T11:41:52Z)
• Resilient VAE: Unsupervised Anomaly Detection at the SLAC Linac Coherent Light Source [3.7390282036618916]
  This paper introduces the Resilient Variational Autoencoder (ResVAE), a deep generative model specifically designed for anomaly detection. ResVAE exhibits resilience to anomalies present in the training data and provides feature-level anomaly attribution. We apply our proposed method to detect anomalies in the accelerator status at the SLAC Linac Coherent Light Source.
  arXiv  Detail & Related papers  (2023-09-05T15:53:41Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• NADS: Neural Architecture Distribution Search for Uncertainty Awareness [79.18710225716791]
  Machine learning (ML) systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a distribution different from training data. Existing OoD detection approaches are prone to errors and even sometimes assign higher likelihoods to OoD samples. We propose Neural Architecture Distribution Search (NADS) to identify common building blocks among all uncertainty-aware architectures.
  arXiv  Detail & Related papers  (2020-06-11T17:39:07Z)
• Anomalous Example Detection in Deep Learning: A Survey [98.2295889723002]
  This survey tries to provide a structured and comprehensive overview of the research on anomaly detection for Deep Learning applications. We provide a taxonomy for existing techniques based on their underlying assumptions and adopted approaches. We highlight the unsolved research challenges while applying anomaly detection techniques in DL systems and present some high-impact future research directions.
  arXiv  Detail & Related papers  (2020-03-16T02:47:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.