Distribution free uncertainty quantification in neuroscience-inspired deep operators

• URL: http://arxiv.org/abs/2412.09369v1
• Date: Thu, 12 Dec 2024 15:37:02 GMT
• Title: Distribution free uncertainty quantification in neuroscience-inspired deep operators
• Authors: Shailesh Garg, Souvik Chakraborty,
• Abstract summary: Energy-efficient deep learning algorithms are essential for a sustainable future and feasible edge computing setups. In this paper, we introduce the Conformalized Randomized Prior Operator (CRP-O) framework to quantify uncertainty in both conventional and spiking neural operators. We show that the conformalized RP-VSWNO significantly enhance UQ estimates compared to vanilla RP-VSWNO, Quantile WNO (Q-WNO), and Conformalized Quantile WNO (CQ-WNO)
• Score: 1.8416014644193066
• License:
• Abstract: Energy-efficient deep learning algorithms are essential for a sustainable future and feasible edge computing setups. Spiking neural networks (SNNs), inspired from neuroscience, are a positive step in the direction of achieving the required energy efficiency. However, in a bid to lower the energy requirements, accuracy is marginally sacrificed. Hence, predictions of such deep learning algorithms require an uncertainty measure that can inform users regarding the bounds of a certain output. In this paper, we introduce the Conformalized Randomized Prior Operator (CRP-O) framework that leverages Randomized Prior (RP) networks and Split Conformal Prediction (SCP) to quantify uncertainty in both conventional and spiking neural operators. To further enable zero-shot super-resolution in UQ, we propose an extension incorporating Gaussian Process Regression. This enhanced super-resolution-enabled CRP-O framework is integrated with the recently developed Variable Spiking Wavelet Neural Operator (VSWNO). To test the performance of the obtained calibrated uncertainty bounds, we discuss four different examples covering both one-dimensional and two-dimensional partial differential equations. Results demonstrate that the uncertainty bounds produced by the conformalized RP-VSWNO significantly enhance UQ estimates compared to vanilla RP-VSWNO, Quantile WNO (Q-WNO), and Conformalized Quantile WNO (CQ-WNO). These findings underscore the potential of the proposed approach for practical applications.

Related papers

• Chaos into Order: Neural Framework for Expected Value Estimation of Stochastic Partial Differential Equations [0.9944647907864256]
  We introduce a novel neural framework for SPDE estimation that eliminates the need for discretization and explicitly modeling uncertainty. This is the first neural framework capable directly estimating the expected values of SPDEs in an entirely non-discretized manner, offering a step forward in scientific computing. Our findings highlight the immense potential of neural-based SPDE solvers, particularly for high-dimensional problems where conventional techniques falter.
  arXiv  Detail & Related papers  (2025-02-05T23:27:28Z)
• RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators [0.0]
  We present Random Projection-based Operator Networks (RandONets) RandONets are shallow networks with random projections that learn linear and nonlinear operators. We show, that for this particular task, RandONets outperform, both in terms of numerical approximation accuracy and computational cost, the vanilla" DeepOnets.
  arXiv  Detail & Related papers  (2024-06-08T13:20:48Z)
• Pointer Networks with Q-Learning for Combinatorial Optimization [55.2480439325792]
  We introduce the Pointer Q-Network (PQN), a hybrid neural architecture that integrates model-free Q-value policy approximation with Pointer Networks (Ptr-Nets) Our empirical results demonstrate the efficacy of this approach, also testing the model in unstable environments.
  arXiv  Detail & Related papers  (2023-11-05T12:03:58Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• Randomized prior wavelet neural operator for uncertainty quantification [0.0]
  We propose a novel data-driven operator learning framework referred to as the textitRandomized Prior Wavelet Neural Operator (RP-WNO) The proposed RP-WNO is an extension of the recently proposed wavelet neural operator, which boasts excellent generalizing capabilities but cannot estimate the uncertainty associated with its predictions.
  arXiv  Detail & Related papers  (2023-02-02T12:28:49Z)
• Density Regression and Uncertainty Quantification with Bayesian Deep Noise Neural Networks [4.376565880192482]
  Deep neural network (DNN) models have achieved state-of-the-art predictive accuracy in a wide range of supervised learning applications. accurately quantifying the uncertainty in DNN predictions remains a challenging task. We propose the Bayesian Deep Noise Neural Network (B-DeepNoise), which generalizes standard Bayesian DNNs by extending the random noise variable to all hidden layers. We evaluate B-DeepNoise against existing methods on benchmark regression datasets, demonstrating its superior performance in terms of prediction accuracy, uncertainty quantification accuracy, and uncertainty quantification efficiency.
  arXiv  Detail & Related papers  (2022-06-12T02:47:29Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
  Implicit neural networks show improved accuracy and significant reduction in memory consumption. They can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks.
  arXiv  Detail & Related papers  (2021-06-06T18:05:02Z)
• Amortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation [99.92568326314667]
  We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation. Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle. We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-11-05T08:04:34Z)
• Chance-Constrained Control with Lexicographic Deep Reinforcement Learning [77.34726150561087]
  This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.
  arXiv  Detail & Related papers  (2020-10-19T13:09:14Z)
• Robustness Analysis of Neural Networks via Efficient Partitioning with Applications in Control Systems [45.35808135708894]
  Neural networks (NNs) are now routinely implemented on systems that must operate in uncertain environments. This paper unifies propagation and partition approaches to provide a family of robustness analysis algorithms. New partitioning techniques are aware of their current bound estimates and desired boundary shape.
  arXiv  Detail & Related papers  (2020-10-01T16:51:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.