Scalable Subsampling Inference for Deep Neural Networks

• URL: http://arxiv.org/abs/2405.08276v1
• Date: Tue, 14 May 2024 02:11:38 GMT
• Title: Scalable Subsampling Inference for Deep Neural Networks
• Authors: Kejin Wu, Dimitris N. Politis,
• Abstract summary: A non-asymptotic error bound has been developed to measure the performance of the fully connected DNN estimator. A non-random subsampling technique--scalable subsampling--is applied to construct a subagged' DNN estimator. The proposed confidence/prediction intervals appear to work well in finite samples.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks (DNN) has received increasing attention in machine learning applications in the last several years. Recently, a non-asymptotic error bound has been developed to measure the performance of the fully connected DNN estimator with ReLU activation functions for estimating regression models. The paper at hand gives a small improvement on the current error bound based on the latest results on the approximation ability of DNN. More importantly, however, a non-random subsampling technique--scalable subsampling--is applied to construct a `subagged' DNN estimator. Under regularity conditions, it is shown that the subagged DNN estimator is computationally efficient without sacrificing accuracy for either estimation or prediction tasks. Beyond point estimation/prediction, we propose different approaches to build confidence and prediction intervals based on the subagged DNN estimator. In addition to being asymptotically valid, the proposed confidence/prediction intervals appear to work well in finite samples. All in all, the scalable subsampling DNN estimator offers the complete package in terms of statistical inference, i.e., (a) computational efficiency; (b) point estimation/prediction accuracy; and (c) allowing for the construction of practically useful confidence and prediction intervals.

Related papers

• Uncertainty in Graph Neural Networks: A Survey [50.63474656037679]
  Graph Neural Networks (GNNs) have been extensively used in various real-world applications. However, the predictive uncertainty of GNNs stemming from diverse sources can lead to unstable and erroneous predictions. This survey aims to provide a comprehensive overview of the GNNs from the perspective of uncertainty.
  arXiv  Detail & Related papers  (2024-03-11T21:54:52Z)
• Uncertainty Quantification over Graph with Conformalized Graph Neural Networks [52.20904874696597]
  Graph Neural Networks (GNNs) are powerful machine learning prediction models on graph-structured data. GNNs lack rigorous uncertainty estimates, limiting their reliable deployment in settings where the cost of errors is significant. We propose conformalized GNN (CF-GNN), extending conformal prediction (CP) to graph-based models for guaranteed uncertainty estimates.
  arXiv  Detail & Related papers  (2023-05-23T21:38:23Z)
• Confidence-Nets: A Step Towards better Prediction Intervals for regression Neural Networks on small datasets [0.0]
  We propose an ensemble method that attempts to estimate the uncertainty of predictions, increase their accuracy and provide an interval for the expected variation. The proposed method is tested on various datasets, and a significant improvement in the performance of the neural network model is seen.
  arXiv  Detail & Related papers  (2022-10-31T06:38:40Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• Adaptive Degradation Process with Deep Learning-Driven Trajectory [5.060233857860902]
  Remaining useful life (RUL) estimation is a crucial component in the implementation of intelligent predictive maintenance and health management. This paper develops a hybrid DNN-based prognostic approach, where a Wiener-based-degradation model is enhanced with adaptive drift to characterize the system degradation. An LSTM-CNN encoder-decoder is developed to predict future degradation trajectories by jointly learning noise coefficients as well as drift coefficients, and adaptive drift is updated via Bayesian inference.
  arXiv  Detail & Related papers  (2021-03-22T06:00:42Z)
• A Biased Graph Neural Network Sampler with Near-Optimal Regret [57.70126763759996]
  Graph neural networks (GNN) have emerged as a vehicle for applying deep network architectures to graph and relational data. In this paper, we build upon existing work and treat GNN neighbor sampling as a multi-armed bandit problem. We introduce a newly-designed reward function that introduces some degree of bias designed to reduce variance and avoid unstable, possibly-unbounded payouts.
  arXiv  Detail & Related papers  (2021-03-01T15:55:58Z)
• Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfaces [23.21591478556582]
  We develop a minimax rate analysis to describe the reason that deep neural networks (DNNs) perform better than other standard methods. This study tries to fill this gap by considering the estimation for a class of non-smooth functions that have singularities on hypersurfaces.
  arXiv  Detail & Related papers  (2020-11-04T12:51:14Z)
• Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions [121.10450359856242]
  Recurrent neural networks (RNNs) are instrumental in modelling sequential and time-series data. Existing approaches for uncertainty quantification in RNNs are based predominantly on Bayesian methods. We develop a frequentist alternative that: (a) does not interfere with model training or compromise its accuracy, (b) applies to any RNN architecture, and (c) provides theoretical coverage guarantees on the estimated uncertainty intervals.
  arXiv  Detail & Related papers  (2020-06-20T22:45:32Z)
• PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction [14.635820704895034]
  We present PIVEN, a deep neural network for producing both a PI and a value prediction. Our approach makes no assumptions regarding data distribution within the PI, making its value prediction more effective for various real-world problems.
  arXiv  Detail & Related papers  (2020-06-09T09:29:58Z)
• Interval Neural Networks: Uncertainty Scores [11.74565957328407]
  We propose a fast, non-Bayesian method for producing uncertainty scores in the output of pre-trained deep neural networks (DNNs) This interval neural network (INN) has interval valued parameters and propagates its input using interval arithmetic. In numerical experiments on an image reconstruction task, we demonstrate the practical utility of INNs as a proxy for the prediction error.
  arXiv  Detail & Related papers  (2020-03-25T18:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.