A Probabilistic Generative Model of Free Categories

• URL: http://arxiv.org/abs/2205.04545v1
• Date: Mon, 9 May 2022 20:35:08 GMT
• Title: A Probabilistic Generative Model of Free Categories
• Authors: Eli Sennesh, Tom Xu, Yoshihiro Maruyama
• Abstract summary: This paper defines a probabilistic generative model of morphisms in free monoidal categories over domain-specific generating objects and morphisms. Acyclic wiring diagrams can model specifications for morphisms, which the model can use to generate morphisms. A concrete experiment shows that the free category achieves prior competitive reconstruction performance on the Omniglot dataset.
• Score: 1.7679374058425343
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Applied category theory has recently developed libraries for computing with morphisms in interesting categories, while machine learning has developed ways of learning programs in interesting languages. Taking the analogy between categories and languages seriously, this paper defines a probabilistic generative model of morphisms in free monoidal categories over domain-specific generating objects and morphisms. The paper shows how acyclic directed wiring diagrams can model specifications for morphisms, which the model can use to generate morphisms. Amortized variational inference in the generative model then enables learning of parameters (by maximum likelihood) and inference of latent variables (by Bayesian inversion). A concrete experiment shows that the free category prior achieves competitive reconstruction performance on the Omniglot dataset.

Related papers

• Combinatorial Complex Score-based Diffusion Modelling through Stochastic Differential Equations [0.0]
  This thesis explores the potential of score-based generative models in generating graphs. In this thesis, we propose a unified framework by employing differential equations. This innovation overcomes limitations in existing frameworks that focus solely on graph generation, opening up new possibilities in generative AI.
  arXiv  Detail & Related papers  (2024-06-07T13:16:10Z)
• Shape Arithmetic Expressions: Advancing Scientific Discovery Beyond Closed-Form Equations [56.78271181959529]
  Generalized Additive Models (GAMs) can capture non-linear relationships between variables and targets, but they cannot capture intricate feature interactions. We propose Shape Expressions Arithmetic ( SHAREs) that fuses GAM's flexible shape functions with the complex feature interactions found in mathematical expressions. We also design a set of rules for constructing SHAREs that guarantee transparency of the found expressions beyond the standard constraints.
  arXiv  Detail & Related papers  (2024-04-15T13:44:01Z)
• Algebraic Dynamical Systems in Machine Learning [0.1843404256219181]
  We show that a function applied to the output of an iterated rewriting system defines a formal class of models. We also show that these algebraic models are a natural language for describing the compositionality of dynamic models. These models provide a template for the generalisation of the above dynamic models to learning problems on structured or non-numerical data.
  arXiv  Detail & Related papers  (2023-11-06T14:10:40Z)
• Language Model Cascades [72.18809575261498]
  Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. Cases with control flow and dynamic structure require techniques from probabilistic programming. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
  arXiv  Detail & Related papers  (2022-07-21T07:35:18Z)
• Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
  In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
  arXiv  Detail & Related papers  (2021-07-07T11:17:09Z)
• Conditional Neural Relational Inference for Interacting Systems [58.141087282927415]
  We learn to model the dynamics of similar yet distinct groups of interacting objects. We develop a model that allows us to do conditional generation from any such group given its vectorial description. We evaluate our model in the setting of modeling human gait and, in particular pathological human gait.
  arXiv  Detail & Related papers  (2021-06-21T13:05:48Z)
• Discovering Relationships between Object Categories via Universal Canonical Maps [80.07703460198198]
  We tackle the problem of learning the geometry of multiple categories of deformable objects jointly. Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects. We show that improved correspondences can be learned automatically as a natural byproduct of learning category-specific dense pose predictors.
  arXiv  Detail & Related papers  (2021-06-17T18:38:18Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• The Role of Isomorphism Classes in Multi-Relational Datasets [6.419762264544509]
  We show that isomorphism leakage overestimates performance in multi-relational inference. We propose isomorphism-aware synthetic benchmarks for model evaluation. We also demonstrate that isomorphism classes can be utilised through a simple prioritisation scheme.
  arXiv  Detail & Related papers  (2020-09-30T12:15:24Z)
• Categorical Stochastic Processes and Likelihood [1.14219428942199]
  We take a Category Theoretic perspective on the relationship between probabilistic modeling and function approximation. We show how these extensions relate to the category Stoch and other Markov Categories. We conclude with a demonstration of how the Maximum Likelihood Estimation procedure defines an identity-on-objects functor from the category of statistical models to the category of learners.
  arXiv  Detail & Related papers  (2020-05-10T18:00:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.