A Falsificationist Account of Artificial Neural Networks
- URL: http://arxiv.org/abs/2205.01421v1
- Date: Tue, 3 May 2022 11:19:56 GMT
- Title: A Falsificationist Account of Artificial Neural Networks
- Authors: Oliver Buchholz and Eric Raidl
- Abstract summary: We argue that the idea of falsification is central to the methodology of machine learning.
Machine learning operates at the intersection of statistics and computer science.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning operates at the intersection of statistics and computer
science. This raises the question as to its underlying methodology. While much
emphasis has been put on the close link between the process of learning from
data and induction, the falsificationist component of machine learning has
received minor attention. In this paper, we argue that the idea of
falsification is central to the methodology of machine learning. It is commonly
thought that machine learning algorithms infer general prediction rules from
past observations. This is akin to a statistical procedure by which estimates
are obtained from a sample of data. But machine learning algorithms can also be
described as choosing one prediction rule from an entire class of functions. In
particular, the algorithm that determines the weights of an artificial neural
network operates by empirical risk minimization and rejects prediction rules
that lack empirical adequacy. It also exhibits a behavior of implicit
regularization that pushes hypothesis choice toward simpler prediction rules.
We argue that taking both aspects together gives rise to a falsificationist
account of artificial neural networks.
Related papers
- Instance-wise Linearization of Neural Network for Model Interpretation [13.583425552511704]
The challenge can dive into the non-linear behavior of the neural network.
For a neural network model, the non-linear behavior is often caused by non-linear activation units of a model.
We propose an instance-wise linearization approach to reformulates the forward computation process of a neural network prediction.
arXiv Detail & Related papers (2023-10-25T02:07:39Z) - Utility-Probability Duality of Neural Networks [4.871730595406078]
We propose an alternative utility-based explanation to the standard supervised learning procedure in deep learning.
The basic idea is to interpret the learned neural network not as a probability model but as an ordinal utility function.
We show that for all neural networks with softmax outputs, the SGD learning dynamic of maximum likelihood estimation can be seen as an iteration process.
arXiv Detail & Related papers (2023-05-24T08:09:07Z) - Algorithmic failure as a humanities methodology: machine learning's
mispredictions identify rich cases for qualitative analysis [0.0]
I trained a simple machine learning algorithm to predict whether or not an action was active or passive using only information about fictional characters.
The results thus support Munk et al.'s theory that failed predictions can be productively used to identify rich cases for qualitative analysis.
Further research is needed to develop an understanding of what kinds of data the method is useful for and which kinds of machine learning are most generative.
arXiv Detail & Related papers (2023-05-19T13:24:32Z) - Prediction-Powered Inference [68.97619568620709]
Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system.
The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients.
Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning.
arXiv Detail & Related papers (2023-01-23T18:59:28Z) - VCNet: A self-explaining model for realistic counterfactual generation [52.77024349608834]
Counterfactual explanation is a class of methods to make local explanations of machine learning decisions.
We present VCNet-Variational Counter Net, a model architecture that combines a predictor and a counterfactual generator.
We show that VCNet is able to both generate predictions, and to generate counterfactual explanations without having to solve another minimisation problem.
arXiv Detail & Related papers (2022-12-21T08:45:32Z) - Multiclass classification utilising an estimated algorithmic probability
prior [0.5156484100374058]
We study how algorithmic information theory, especially algorithmic probability, may aid in a machine learning task.
This work is one of the first to demonstrate how algorithmic probability can aid in a concrete, real-world, machine learning problem.
arXiv Detail & Related papers (2022-12-14T07:50:12Z) - Refining neural network predictions using background knowledge [68.35246878394702]
We show we can use logical background knowledge in learning system to compensate for a lack of labeled training data.
We introduce differentiable refinement functions that find a corrected prediction close to the original prediction.
This algorithm finds optimal refinements on complex SAT formulas in significantly fewer iterations and frequently finds solutions where gradient descent can not.
arXiv Detail & Related papers (2022-06-10T10:17:59Z) - Scalable computation of prediction intervals for neural networks via
matrix sketching [79.44177623781043]
Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure.
This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
arXiv Detail & Related papers (2022-05-06T13:18:31Z) - Hessian-based toolbox for reliable and interpretable machine learning in
physics [58.720142291102135]
We present a toolbox for interpretability and reliability, extrapolation of the model architecture.
It provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an agnostic score for the model predictions.
Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.
arXiv Detail & Related papers (2021-08-04T16:32:59Z) - Vulnerability Under Adversarial Machine Learning: Bias or Variance? [77.30759061082085]
We investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network.
Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation.
We introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies.
arXiv Detail & Related papers (2020-08-01T00:58:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.