Uncertainty Estimation in Machine Learning
- URL: http://arxiv.org/abs/2206.01749v1
- Date: Fri, 3 Jun 2022 16:11:11 GMT
- Title: Uncertainty Estimation in Machine Learning
- Authors: Valentin Arkov
- Abstract summary: In machine learning the model complexity and severe nonlinearity become serious obstacles to uncertainty evaluation.
The latest example of a pre-trained model is the Generative Pre-trained Transformer 3 with hundreds of billions of parameters and a half-terabyte training dataset.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Most machine learning techniques are based upon statistical learning theory,
often simplified for the sake of computing speed. This paper is focused on the
uncertainty aspect of mathematical modeling in machine learning. Regression
analysis is chosen to further investigate the evaluation aspect of uncertainty
in model coefficients and, more importantly, in the output feature value
predictions. A survey demonstrates major stages in the conventional least
squares approach to the creation of the regression model, along with its
uncertainty estimation. On the other hand, it is shown that in machine learning
the model complexity and severe nonlinearity become serious obstacles to
uncertainty evaluation. Furthermore, the process of machine model training
demands high computing power, not available at the level of personal computers.
This is why so-called pre-trained models are widely used in such areas of
machine learning as natural language processing. The latest example of a
pre-trained model is the Generative Pre-trained Transformer 3 with hundreds of
billions of parameters and a half-terabyte training dataset. Similarly,
mathematical models built from real data are growing in complexity which is
accompanied by the growing amount of training data. However, when machine
models and their predictions are used in decision-making, one needs to estimate
uncertainty and evaluate accompanying risks. This problem could be resolved
with non-parametric techniques at the expense of greater demand for computing
power, which can be offered by modern supercomputers available, including those
utilizing graphical and tensor processing units along with the conventional
central processors.
Related papers
- Scalable Higher-Order Tensor Product Spline Models [0.0]
We propose a new approach using a factorization method to derive a highly scalable higher-order tensor product spline model.
Our method allows for the incorporation of all (higher-order) interactions of non-linear feature effects while having computational costs proportional to a model without interactions.
arXiv Detail & Related papers (2024-02-02T01:18:48Z) - Zero-knowledge Proof Meets Machine Learning in Verifiability: A Survey [19.70499936572449]
High-quality models rely not only on efficient optimization algorithms but also on the training and learning processes built upon vast amounts of data and computational power.
Due to various challenges such as limited computational resources and data privacy concerns, users in need of models often cannot train machine learning models locally.
This paper presents a comprehensive survey of zero-knowledge proof-based verifiable machine learning (ZKP-VML) technology.
arXiv Detail & Related papers (2023-10-23T12:15:23Z) - Stabilizing Machine Learning Prediction of Dynamics: Noise and
Noise-inspired Regularization [58.720142291102135]
Recent has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of chaotic dynamical systems.
In the absence of mitigating techniques, this technique can result in artificially rapid error growth, leading to inaccurate predictions and/or climate instability.
We introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training.
arXiv Detail & Related papers (2022-11-09T23:40:52Z) - Synthetic Model Combination: An Instance-wise Approach to Unsupervised
Ensemble Learning [92.89846887298852]
Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data.
Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
arXiv Detail & Related papers (2022-10-11T10:20:31Z) - A review of predictive uncertainty estimation with machine learning [0.0]
We review the topic of predictive uncertainty estimation with machine learning algorithms.
We discuss the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions.
The review expedites our understanding on how to develop new algorithms tailored to users' needs.
arXiv Detail & Related papers (2022-09-17T10:36:30Z) - Advancing Reacting Flow Simulations with Data-Driven Models [50.9598607067535]
Key to effective use of machine learning tools in multi-physics problems is to couple them to physical and computer models.
The present chapter reviews some of the open opportunities for the application of data-driven reduced-order modeling of combustion systems.
arXiv Detail & Related papers (2022-09-05T16:48:34Z) - Additive Higher-Order Factorization Machines [0.0]
We derive a scalable high-order tensor product spline model using a factorization approach.
Our method allows to include all (higher-order) interactions of non-linear feature effects.
We prove both theoretically and empirically that our methods scales notably better than existing approaches.
arXiv Detail & Related papers (2022-05-28T19:50:52Z) - Hessian-based toolbox for reliable and interpretable machine learning in
physics [58.720142291102135]
We present a toolbox for interpretability and reliability, extrapolation of the model architecture.
It provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an agnostic score for the model predictions.
Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.
arXiv Detail & Related papers (2021-08-04T16:32:59Z) - Using Data Assimilation to Train a Hybrid Forecast System that Combines
Machine-Learning and Knowledge-Based Components [52.77024349608834]
We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is noisy partial measurements.
We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.
arXiv Detail & Related papers (2021-02-15T19:56:48Z) - Model-Based Deep Learning [155.063817656602]
Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques.
Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance.
We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
arXiv Detail & Related papers (2020-12-15T16:29:49Z) - Introduction to Rare-Event Predictive Modeling for Inferential
Statisticians -- A Hands-On Application in the Prediction of Breakthrough
Patents [0.0]
We introduce a machine learning (ML) approach to quantitative analysis geared towards optimizing the predictive performance.
We discuss the potential synergies between the two fields against the backdrop of this, at first glance, target-incompatibility.
We are providing a hands-on predictive modeling introduction for a quantitative social science audience while aiming at demystifying computer science jargon.
arXiv Detail & Related papers (2020-03-30T13:06:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.