Computing with Categories in Machine Learning

• URL: http://arxiv.org/abs/2303.04156v1
• Date: Tue, 7 Mar 2023 17:26:18 GMT
• Title: Computing with Categories in Machine Learning
• Authors: Eli Sennesh, Tom Xu, Yoshihiro Maruyama
• Abstract summary: We introduce DisCoPyro as a categorical structure learning framework. DisCoPyro combines categorical structures with amortized variational inference. We speculate that DisCoPyro could ultimately contribute to the development of artificial general intelligence.
• Score: 1.7679374058425343
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Category theory has been successfully applied in various domains of science, shedding light on universal principles unifying diverse phenomena and thereby enabling knowledge transfer between them. Applications to machine learning have been pursued recently, and yet there is still a gap between abstract mathematical foundations and concrete applications to machine learning tasks. In this paper we introduce DisCoPyro as a categorical structure learning framework, which combines categorical structures (such as symmetric monoidal categories and operads) with amortized variational inference, and can be applied, e.g., in program learning for variational autoencoders. We provide both mathematical foundations and concrete applications together with comparison of experimental performance with other models (e.g., neuro-symbolic models). We speculate that DisCoPyro could ultimately contribute to the development of artificial general intelligence.

Related papers

• Towards a Categorical Foundation of Deep Learning: A Survey [0.0]
  This thesis is a survey that covers some recent work attempting to study machine learning categorically. acting as a lingua franca of mathematics and science, category theory might be able to give a unifying structure to the field of machine learning.
  arXiv  Detail & Related papers  (2024-10-07T13:11:16Z)
• Symmetry-Enriched Learning: A Category-Theoretic Framework for Robust Machine Learning Models [0.0]
  We introduce new mathematical constructs, including hyper-symmetry categories and functorial representations, to model complex transformations within machine learning algorithms. Our contributions include the design of symmetry-enriched learning models, the development of advanced optimization techniques leveraging categorical symmetries, and the theoretical analysis of their implications for model robustness, generalization, and convergence.
  arXiv  Detail & Related papers  (2024-09-18T16:20:57Z)
• Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks [50.29356570858905]
  We introduce the Dynamical Systems Framework (DSF), which allows a principled investigation of all these architectures in a common representation. We provide principled comparisons between softmax attention and other model classes, discussing the theoretical conditions under which softmax attention can be approximated. This shows the DSF's potential to guide the systematic development of future more efficient and scalable foundation models.
  arXiv  Detail & Related papers  (2024-05-24T17:19:57Z)
• Token Space: A Category Theory Framework for AI Computations [0.0]
  This paper introduces the Token Space framework, a novel mathematical construct designed to enhance the interpretability and effectiveness of deep learning models. By establishing a categorical structure at the Token level, we provide a new lens through which AI computations can be understood.
  arXiv  Detail & Related papers  (2024-04-11T15:56:06Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• A Review of Neuroscience-Inspired Machine Learning [58.72729525961739]
  Bio-plausible credit assignment is compatible with practically any learning condition and is energy-efficient. In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks. We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.
  arXiv  Detail & Related papers  (2024-02-16T18:05:09Z)
• Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions [68.6358773622615]
  This paper provides an overview of the computational and theoretical foundations of multimodal machine learning. We propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification. Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches.
  arXiv  Detail & Related papers  (2022-09-07T19:21:19Z)
• Symmetry Group Equivariant Architectures for Physics [52.784926970374556]
  In the domain of machine learning, an awareness of symmetries has driven impressive performance breakthroughs. We argue that both the physics community and the broader machine learning community have much to understand.
  arXiv  Detail & Related papers  (2022-03-11T18:27:04Z)
• Panoramic Learning with A Standardized Machine Learning Formalism [116.34627789412102]
  This paper presents a standardized equation of the learning objective, that offers a unifying understanding of diverse ML algorithms. It also provides guidance for mechanic design of new ML solutions, and serves as a promising vehicle towards panoramic learning with all experiences.
  arXiv  Detail & Related papers  (2021-08-17T17:44:38Z)
• Category Theory in Machine Learning [1.6758573326215689]
  We document the motivations, goals and common themes across applications of category theory in machine learning. We touch on gradient-based learning, probability, and equivariant learning.
  arXiv  Detail & Related papers  (2021-06-13T15:58:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.