From Approximation to Emergence: A Theory of Deep Learning

• URL: http://arxiv.org/abs/2607.01311v1
• Date: Wed, 01 Jul 2026 17:26:19 GMT
• Title: From Approximation to Emergence: A Theory of Deep Learning
• Abstract summary: From Approximation to Emergence is a unified, proof-oriented account of modern deep learning theory.<n>Written for researchers, graduate students, and mathematically trained practitioners, it offers a rigorous map of deep learning theory.
• Score: 5.863264019032883
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning has outgrown any single mathematical explanation. From Approximation to Emergence develops a unified, proof-oriented account of modern deep learning theory, tracing a path from the classical foundations of approximation, optimization, and generalization to the contemporary mechanisms of overparameterization, robustness, generative modeling, transformers, in-context learning, scaling laws, interpretability, alignment, and emergence. Rather than presenting isolated results, the book organizes a broad literature into a coherent research narrative: each theory is examined through the object it controls, the assumptions that make it valid, and the phenomena it leaves unexplained. Written for researchers, graduate students, and mathematically trained practitioners, this monograph offers a rigorous map of deep learning theory as it stands today: powerful, incomplete, and increasingly centered on the question of how learned mechanisms arise from scale, data, architecture, and training.

Related papers

• From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery [60.11725132433485]
  We argue that scientific discovery is fundamentally a problem of knowledge organisation.<n>We introduce Mechanistic World Models, a new design paradigm that places reusable mechanisms at the centre of representation, computation and learning.
  arXiv  Detail & Related papers  (2026-07-14T08:03:27Z)
• There Will Be a Scientific Theory of Deep Learning [62.690977616250954]
  We make the case that a scientific theory of deep learning is emerging.<n>We identify five growing bodies of work that point toward such a theory.<n>We argue that the emerging theory is best thought of as a mechanics of the learning process.
  arXiv  Detail & Related papers  (2026-04-23T13:58:12Z)
• Tutorial on the Probabilistic Unification of Estimation Theory, Machine Learning, and Generative AI [0.0]
  This survey presents a unified mathematical framework that connects classical estimation theory, statistical inference, and modern machine learning.<n>We show how techniques such as maximum likelihood estimation, Bayesian inference, and attention mechanisms address uncertainty.<n>It serves as both a theoretical synthesis and a practical guide for students and researchers navigating the evolving landscape of machine learning.
  arXiv  Detail & Related papers  (2025-08-21T16:57:33Z)
• Sheaf theory: from deep geometry to deep learning [0.3749861135832073]
  This paper provides an overview of the applications of sheaf theory in deep learning, data science, and computer science.<n>We describe intuitions and motivations underlying sheaf theory shared by both theoretical researchers and practitioners.<n>We present a new algorithm to compute sheaf cohomology on arbitrary finite posets in response.
  arXiv  Detail & Related papers  (2025-02-21T14:00:25Z)
• Information-Theoretic Foundations for Machine Learning [20.617552198581024]
  We propose a theoretical framework which attempts to provide rigor to existing practices in machine learning.<n>We provide a framework rooted in Bayesian statistics and Shannon's information theory which is general enough to unify the analysis of many phenomena in machine learning.<n>Unlike existing analyses that weaken with increasing data complexity, our theoretical tools provide accurate insights across diverse machine learning settings.
  arXiv  Detail & Related papers  (2024-07-17T03:18:40Z)
• Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
  Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models.<n>We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model.<n>We substantiate our theoretical claims with synthetic data experiments.
  arXiv  Detail & Related papers  (2024-06-01T18:01:03Z)
• Neural Causal Abstractions [63.21695740637627]
  We develop a new family of causal abstractions by clustering variables and their domains. We show that such abstractions are learnable in practical settings through Neural Causal Models. Our experiments support the theory and illustrate how to scale causal inferences to high-dimensional settings involving image data.
  arXiv  Detail & Related papers  (2024-01-05T02:00:27Z)
• Causal Reasoning Meets Visual Representation Learning: A Prospective Study [117.08431221482638]
  Lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. Inspired by the strong inference ability of human-level agents, recent years have witnessed great effort in developing causal reasoning paradigms. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods.
  arXiv  Detail & Related papers  (2022-04-26T02:22:28Z)
• Envisioning Future Deep Learning Theories: Some Basic Concepts and Characteristics [30.365274034429508]
  We argue that a future deep learning theory should inherit three characteristics: a textitarchhierically structured network architecture, parameters textititeratively optimized using gradient-based methods, and information from the data that evolves textitcompressively We integrate these characteristics into a graphical model called textitneurashed, which effectively explains some common empirical patterns in deep learning.
  arXiv  Detail & Related papers  (2021-12-17T19:51:26Z)
• Recent advances in deep learning theory [104.01582662336256]
  This paper reviews and organizes the recent advances in deep learning theory. The literature is categorized in six groups: (1) complexity and capacity-based approaches for analysing the generalizability of deep learning; (2) differential equations and their dynamic systems for modelling gradient descent and its variants; (3) the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems; and (5) theoretical foundations of several special structures in network architectures.
  arXiv  Detail & Related papers  (2020-12-20T14:16:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.