Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustness

• URL: http://arxiv.org/abs/2307.02454v1
• Date: Wed, 5 Jul 2023 17:27:17 GMT
• Title: Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustness
• Authors: Carsten Hartmann, Lorenz Richter
• Abstract summary: The recent advances in machine learning in various fields of applications can be attributed to the rise of deep learning (DL) methods and architectures. Despite being a key technology behind autonomous cars, image processing, speech recognition, etc., a notorious problem remains the lack of theoretical understanding of DL.
• Score: 3.1219977244201056
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The recent advances in machine learning in various fields of applications can be largely attributed to the rise of deep learning (DL) methods and architectures. Despite being a key technology behind autonomous cars, image processing, speech recognition, etc., a notorious problem remains the lack of theoretical understanding of DL and related interpretability and (adversarial) robustness issues. Understanding the specifics of DL, as compared to, say, other forms of nonlinear regression methods or statistical learning, is interesting from a mathematical perspective, but at the same time it is of crucial importance in practice: treating neural networks as mere black boxes might be sufficient in certain cases, but many applications require waterproof performance guarantees and a deeper understanding of what could go wrong and why it could go wrong. It is probably fair to say that, despite being mathematically well founded as a method to approximate complicated functions, DL is mostly still more like modern alchemy that is firmly in the hands of engineers and computer scientists. Nevertheless, it is evident that certain specifics of DL that could explain its success in applications demands systematic mathematical approaches. In this work, we review robustness issues of DL and particularly bridge concerns and attempts from approximation theory to statistical learning theory. Further, we review Bayesian Deep Learning as a means for uncertainty quantification and rigorous explainability.

Related papers

• Computability of Classification and Deep Learning: From Theoretical Limits to Practical Feasibility through Quantization [53.15874572081944]
  We study computability in the deep learning framework from two perspectives. We show algorithmic limitations in training deep neural networks even in cases where the underlying problem is well-behaved. Finally, we show that in quantized versions of classification and deep network training, computability restrictions do not arise or can be overcome to a certain degree.
  arXiv  Detail & Related papers  (2024-08-12T15:02:26Z)
• Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirement [65.26723285209853]
  We derive a framework to analyze whether a transparent implementation in a computing model is feasible. Based on previous results, we find that Blum-Shub-Smale Machines have the potential to establish trustworthy solvers for inverse problems.
  arXiv  Detail & Related papers  (2024-01-18T15:32:38Z)
• Applying statistical learning theory to deep learning [21.24637996678039]
  The goal of these lectures is to provide an overview of some of the main questions that arise when attempting to understand deep learning. We discuss implicit bias in the context of benign overfitting. We provide a detailed study of the implicit bias of gradient descent on linear diagonal networks for various regression tasks.
  arXiv  Detail & Related papers  (2023-11-26T20:00:53Z)
• Dual Box Embeddings for the Description Logic EL++ [16.70961576041243]
  Similar to Knowledge Graphs (KGs), Knowledge Graphs are often incomplete, and maintaining and constructing them has proved challenging. Similar to KGs, a promising approach is to learn embeddings in a latent vector space, while additionally ensuring they adhere to the semantics of the underlying DL. We propose a novel ontology embedding method named Box$2$EL for the DL EL++, which represents both concepts and roles as boxes.
  arXiv  Detail & Related papers  (2023-01-26T14:13:37Z)
• Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics [77.34726150561087]
  Recent developments in artificial neural networks, particularly deep learning (DL), are reviewed in detail. Both hybrid and pure machine learning (ML) methods are discussed. History and limitations of AI are recounted and discussed, with particular attention at pointing out misstatements or misconceptions of the classics.
  arXiv  Detail & Related papers  (2022-12-18T02:03:00Z)
• Scalable approach to many-body localization via quantum data [69.3939291118954]
  Many-body localization is a notoriously difficult phenomenon from quantum many-body physics. We propose a flexible neural network based learning approach that circumvents any computationally expensive step. Our approach can be applied to large-scale quantum experiments to provide new insights into quantum many-body physics.
  arXiv  Detail & Related papers  (2022-02-17T19:00:09Z)
• A Reflection on Learning from Data: Epistemology Issues and Limitations [1.8047694351309205]
  This paper reflects on some issues and some limitations of the knowledge discovered in data. The paper sheds some light on the shortcomings of using generic mathematical theories to describe the process. It further highlights the need for theories specialized in learning from data.
  arXiv  Detail & Related papers  (2021-07-28T11:05:34Z)
• Constrained Learning with Non-Convex Losses [119.8736858597118]
  Though learning has become a core technology of modern information processing, there is now ample evidence that it can lead to biased, unsafe, and prejudiced solutions.
  arXiv  Detail & Related papers  (2021-03-08T23:10:33Z)
• Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable? [4.2111286819721485]
  Recent innovations in deep learning (DL) have shown enormous potential to impact individuals and society. Black-Box nature of DL models and over-reliance on massive amounts of data poses challenges for interpretability and explainability of the system. This article demonstrates how knowledge, provided as a knowledge graph, is incorporated into DL methods using knowledge-infused learning.
  arXiv  Detail & Related papers  (2020-10-16T22:55:23Z)
• Deep Q-Learning: Theoretical Insights from an Asymptotic Analysis [3.9871041399267613]
  Deep Q-Learning is an important reinforcement learning algorithm, which involves training a deep neural network to approximate the well-known Q-function. Although wildly successful under laboratory conditions, serious gaps between theory and practice as well as a lack of formal guarantees prevent its use in the real world. We provide a theoretical analysis of a popular version of Deep Q-Learning under realistic verifiable assumptions.
  arXiv  Detail & Related papers  (2020-08-25T07:59:20Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.