Category-Theoretical and Topos-Theoretical Frameworks in Machine Learning: A Survey

• URL: http://arxiv.org/abs/2408.14014v2
• Date: Thu, 29 Aug 2024 06:04:57 GMT
• Title: Category-Theoretical and Topos-Theoretical Frameworks in Machine Learning: A Survey
• Authors: Yiyang Jia, Guohong Peng, Zheng Yang, Tianhao Chen,
• Abstract summary: We provide an overview of category theory-derived machine learning from four mainstream perspectives. For the first three topics, we primarily review research in the past five years, updating and expanding on the previous survey. The fourth topic, which delves into higher category theory, particularly topos theory, is surveyed for the first time in this paper.
• Score: 4.686566164138397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this survey, we provide an overview of category theory-derived machine learning from four mainstream perspectives: gradient-based learning, probability-based learning, invariance and equivalence-based learning, and topos-based learning. For the first three topics, we primarily review research in the past five years, updating and expanding on the previous survey by Shiebler et al.. The fourth topic, which delves into higher category theory, particularly topos theory, is surveyed for the first time in this paper. In certain machine learning methods, the compositionality of functors plays a vital role, prompting the development of specific categorical frameworks. However, when considering how the global properties of a network reflect in local structures and how geometric properties are expressed with logic, the topos structure becomes particularly significant and profound.

Related papers

• Towards a Categorical Foundation of Deep Learning: A Survey [0.0]
  This thesis is a survey that covers some recent work attempting to study machine learning categorically. acting as a lingua franca of mathematics and science, category theory might be able to give a unifying structure to the field of machine learning.
  arXiv  Detail & Related papers  (2024-10-07T13:11:16Z)
• Foundations and Frontiers of Graph Learning Theory [81.39078977407719]
  Recent advancements in graph learning have revolutionized the way to understand and analyze data with complex structures. Graph Neural Networks (GNNs), i.e. neural network architectures designed for learning graph representations, have become a popular paradigm. This article provides a comprehensive summary of the theoretical foundations and breakthroughs concerning the approximation and learning behaviors intrinsic to prevalent graph learning models.
  arXiv  Detail & Related papers  (2024-07-03T14:07:41Z)
• Token Space: A Category Theory Framework for AI Computations [0.0]
  This paper introduces the Token Space framework, a novel mathematical construct designed to enhance the interpretability and effectiveness of deep learning models. By establishing a categorical structure at the Token level, we provide a new lens through which AI computations can be understood.
  arXiv  Detail & Related papers  (2024-04-11T15:56:06Z)
• Fundamental Components of Deep Learning: A category-theoretic approach [0.0]
  This thesis develops a novel mathematical foundation for deep learning based on the language of category theory. We also systematise many existing approaches, placing many existing constructions and concepts under the same umbrella.
  arXiv  Detail & Related papers  (2024-03-13T01:29:40Z)
• Position: Categorical Deep Learning is an Algebraic Theory of All Architectures [1.772996392520906]
  We present our position on the quest for a general-purpose framework for specifying and studying deep learning architectures. We propose to apply category theory as a single theory elegantly subsuming both of these flavours of neural network design. We show how this theory recovers constraints induced by geometric deep learning, as well as implementations of many architectures drawn from the diverse landscape of neural networks.
  arXiv  Detail & Related papers  (2024-02-23T14:01:53Z)
• A Category-theoretical Meta-analysis of Definitions of Disentanglement [97.34033555407403]
  Disentangling the factors of variation in data is a fundamental concept in machine learning. This paper presents a meta-analysis of existing definitions of disentanglement.
  arXiv  Detail & Related papers  (2023-05-11T15:24:20Z)
• Rank Diminishing in Deep Neural Networks [71.03777954670323]
  Rank of neural networks measures information flowing across layers. It is an instance of a key structural condition that applies across broad domains of machine learning. For neural networks, however, the intrinsic mechanism that yields low-rank structures remains vague and unclear.
  arXiv  Detail & Related papers  (2022-06-13T12:03:32Z)
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [50.22269760171131]
  The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. This text is concerned with exposing pre-defined regularities through unified geometric principles. It provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers.
  arXiv  Detail & Related papers  (2021-04-27T21:09:51Z)
• Recent advances in deep learning theory [104.01582662336256]
  This paper reviews and organizes the recent advances in deep learning theory. The literature is categorized in six groups: (1) complexity and capacity-based approaches for analysing the generalizability of deep learning; (2) differential equations and their dynamic systems for modelling gradient descent and its variants; (3) the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems; and (5) theoretical foundations of several special structures in network architectures.
  arXiv  Detail & Related papers  (2020-12-20T14:16:41Z)
• A Survey on Concept Factorization: From Shallow to Deep Representation Learning [104.78577405792592]
  Concept Factorization (CF) has attracted a great deal of interests in the areas of machine learning and data mining. We first re-view the root CF method, and then explore the advancement of CF-based representation learning. We also introduce the potential application areas of CF-based methods.
  arXiv  Detail & Related papers  (2020-07-31T04:19:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.