Categories of Differentiable Polynomial Circuits for Machine Learning

• URL: http://arxiv.org/abs/2203.06430v1
• Date: Sat, 12 Mar 2022 13:03:30 GMT
• Title: Categories of Differentiable Polynomial Circuits for Machine Learning
• Authors: Paul Wilson, Fabio Zanasi
• Abstract summary: We study presentations by generators and equations of classes of RDCs. We propose emphpolynomial circuits as a suitable machine learning model.
• Score: 0.76146285961466
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reverse derivative categories (RDCs) have recently been shown to be a suitable semantic framework for studying machine learning algorithms. Whereas emphasis has been put on training methodologies, less attention has been devoted to particular \emph{model classes}: the concrete categories whose morphisms represent machine learning models. In this paper we study presentations by generators and equations of classes of RDCs. In particular, we propose \emph{polynomial circuits} as a suitable machine learning model. We give an axiomatisation for these circuits and prove a functional completeness result. Finally, we discuss the use of polynomial circuits over specific semirings to perform machine learning with discrete values.

Related papers

• An Approximation Theory Perspective on Machine Learning [1.2289361708127877]
  We will discuss emerging trends in machine learning including the role of shallow/deep networks.<n>Despite function approximation being a fundamental problem in machine learning, approximation theory does not play a central role in the theoretical foundations of the field.
  arXiv  Detail & Related papers  (2025-06-02T18:50:18Z)
• Deep Learning with Parametric Lenses [0.3645042846301408]
  We propose a categorical semantics for machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework. We demonstrate the practical significance of our framework with an implementation in Python.
  arXiv  Detail & Related papers  (2024-03-30T16:34:28Z)
• Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models [55.19497659895122]
  We introduce methods for discovering and applying sparse feature circuits. These are causally implicatedworks of human-interpretable features for explaining language model behaviors.
  arXiv  Detail & Related papers  (2024-03-28T17:56:07Z)
• CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra [62.37017125812101]
  We propose a simple but general framework for large-scale linear algebra problems in machine learning, named CoLA. By combining a linear operator abstraction with compositional dispatch rules, CoLA automatically constructs memory and runtime efficient numerical algorithms. We showcase its efficacy across a broad range of applications, including partial differential equations, Gaussian processes, equivariant model construction, and unsupervised learning.
  arXiv  Detail & Related papers  (2023-09-06T14:59:38Z)
• A Mathematical Model for Curriculum Learning for Parities [8.522887729678637]
  We introduce a CL model for learning the class of k-parities on d bits of a binary string with a neural network trained by gradient descent. We show that a wise choice of training examples involving two or more product distributions, allows to reduce significantly the computational cost of learning this class of functions. We also show that for another class of functions - namely the Hamming mixtures' - CL strategies involving a bounded number of product distributions are not beneficial.
  arXiv  Detail & Related papers  (2023-01-31T18:25:36Z)
• An Introduction to Kernel and Operator Learning Methods for Homogenization by Self-consistent Clustering Analysis [0.48747801442240574]
  The article presents a thorough analysis on the mathematical underpinnings of the operator learning paradigm. The proposed kernel operator learning method uses graph kernel networks to come up with a mechanistic reduced order method for multiscale homogenization.
  arXiv  Detail & Related papers  (2022-12-01T02:36:16Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• A Probabilistic Generative Model of Free Categories [1.7679374058425343]
  This paper defines a probabilistic generative model of morphisms in free monoidal categories over domain-specific generating objects and morphisms. Acyclic wiring diagrams can model specifications for morphisms, which the model can use to generate morphisms. A concrete experiment shows that the free category achieves prior competitive reconstruction performance on the Omniglot dataset.
  arXiv  Detail & Related papers  (2022-05-09T20:35:08Z)
• Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits [0.0]
  We introduce Reverse Derivative Ascent: a categorical analogue of gradient based methods for machine learning. Our motivating example is reverse circuits: we show how our algorithm can be applied to such circuits by using the theory of reverse differential categories. We demonstrate its empirical value by giving experimental results on benchmark machine learning datasets.
  arXiv  Detail & Related papers  (2021-01-26T00:07:20Z)
• Learning outside the Black-Box: The pursuit of interpretable models [78.32475359554395]
  This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function. Our interpretation represents a leap forward from the previous state of the art.
  arXiv  Detail & Related papers  (2020-11-17T12:39:44Z)
• Stochastic Flows and Geometric Optimization on the Orthogonal Group [52.50121190744979]
  We present a new class of geometrically-driven optimization algorithms on the orthogonal group $O(d)$. We show that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, flows and metric learning.
  arXiv  Detail & Related papers  (2020-03-30T15:37:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.