Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't

• URL: http://arxiv.org/abs/2009.10713v3
• Date: Mon, 7 Dec 2020 23:36:20 GMT
• Title: Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
• Authors: Weinan E, Chao Ma, Stephan Wojtowytsch and Lei Wu
• Abstract summary: This article reviews the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also the insight we have gained from careful numerical experiments.
• Score: 11.447492237545788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.

Related papers

• Machine learning and information theory concepts towards an AI Mathematician [77.63761356203105]
  The current state-of-the-art in artificial intelligence is impressive, especially in terms of mastery of language, but not so much in terms of mathematical reasoning. This essay builds on the idea that current deep learning mostly succeeds at system 1 abilities. It takes an information-theoretical posture to ask questions about what constitutes an interesting mathematical statement.
  arXiv  Detail & Related papers  (2024-03-07T15:12:06Z)
• Deep Learning and Geometric Deep Learning: an introduction for mathematicians and physicists [0.0]
  We discuss the inner functioning of the new and successfull algorithms of Deep Learning and Geometric Deep Learning. We go over the key ingredients for these algorithms: the score and loss function and we explain the main steps for the training of a model. We provide some appendices to complement our treatment discussing Kullback-Leibler divergence, regression, Multi-layer Perceptrons and the Universal Approximation Theorem.
  arXiv  Detail & Related papers  (2023-05-09T16:50:36Z)
• Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning models [0.424243593213882]
  It is unclear whether deep learning models possess an elementary understanding of quantities and symbolic numbers. We critically examine the recent literature, concluding that even state-of-the-art architectures often fall short when probed with relatively simple tasks designed to test basic numerical and arithmetic knowledge.
  arXiv  Detail & Related papers  (2023-03-14T09:30:52Z)
• A Survey of Deep Learning for Mathematical Reasoning [71.88150173381153]
  We review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. Recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning.
  arXiv  Detail & Related papers  (2022-12-20T18:46:16Z)
• Deep Learning to See: Towards New Foundations of Computer Vision [88.69805848302266]
  This book criticizes the supposed scientific progress in the field of computer vision. It proposes the investigation of vision within the framework of information-based laws of nature.
  arXiv  Detail & Related papers  (2022-06-30T15:20:36Z)
• The Physics of Machine Learning: An Intuitive Introduction for the Physical Scientist [0.0]
  This article is intended for physical scientists who wish to gain deeper insights into machine learning algorithms. We begin with a review of two energy-based machine learning algorithms, Hopfield networks and Boltzmann machines, and their connection to the Ising model. We then delve into additional, more "practical," machine learning architectures including feedforward neural networks, convolutional neural networks, and autoencoders.
  arXiv  Detail & Related papers  (2021-11-27T15:12:42Z)
• Optimism in the Face of Adversity: Understanding and Improving Deep Learning through Adversarial Robustness [63.627760598441796]
  We provide an in-depth review of the field of adversarial robustness in deep learning. We highlight the intuitive connection between adversarial examples and the geometry of deep neural networks. We provide an overview of the main emerging applications of adversarial robustness beyond security.
  arXiv  Detail & Related papers  (2020-10-19T16:03:46Z)
• Machine Learning and Computational Mathematics [8.160343645537106]
  We discuss how machine learning has already impacted and will further impact computational mathematics, scientific computing and computational science. We describe some of the most important progress that has been made on these issues. Our hope is to put things into a perspective that will help to integrate machine learning with computational mathematics.
  arXiv  Detail & Related papers  (2020-09-23T23:16:46Z)
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives [67.34875595325597]
  In this primer we review the budding field of motion capture with deep learning. We will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists.
  arXiv  Detail & Related papers  (2020-09-01T16:51:33Z)
• Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning [95.18337034090648]
  We propose a dataset, Machine Number Sense (MNS), consisting of visual arithmetic problems automatically generated using a grammar model--And-Or Graph (AOG) These visual arithmetic problems are in the form of geometric figures. We benchmark the MNS dataset using four predominant neural network models as baselines in this visual reasoning task.
  arXiv  Detail & Related papers  (2020-04-25T17:14:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.