Stochastic Neural Network Symmetrisation in Markov Categories

• URL: http://arxiv.org/abs/2406.11814v5
• Date: Thu, 09 Jan 2025 12:14:23 GMT
• Title: Stochastic Neural Network Symmetrisation in Markov Categories
• Authors: Rob Cornish,
• Abstract summary: We consider the problem of symmetrising a neural network along a group homomorphism. We formulate this in terms of Markov categories. We obtain a flexible and compositional framework for symmetrisation.
• Score: 2.0668277618112203
• License:
• Abstract: We consider the problem of symmetrising a neural network along a group homomorphism: given a homomorphism $\varphi : H \to G$, we would like a procedure that converts $H$-equivariant neural networks to $G$-equivariant ones. We formulate this in terms of Markov categories, which allows us to consider neural networks whose outputs may be stochastic, but with measure-theoretic details abstracted away. We obtain a flexible and compositional framework for symmetrisation that relies on minimal assumptions about the structure of the group and the underlying neural network architecture. Our approach recovers existing canonicalisation and averaging techniques for symmetrising deterministic models, and extends to provide a novel methodology for symmetrising stochastic models also. Beyond this, our findings also demonstrate the utility of Markov categories for addressing complex problems in machine learning in a conceptually clear yet mathematically precise way.

Related papers

• Group Crosscoders for Mechanistic Analysis of Symmetry [0.0]
  Group crosscoders systematically discover and analyse symmetrical features in neural networks. We show that group crosscoders can provide systematic insights into how neural networks represent symmetry.
  arXiv  Detail & Related papers  (2024-10-31T17:47:01Z)
• SymDiff: Equivariant Diffusion via Stochastic Symmetrisation [28.614292092399563]
  We propose a novel method for constructing equivariant diffusion models using the recently introduced framework of symmetrisation. SymDiff resembles a learned data augmentation that is deployed at sampling time, and is lightweight, computationally efficient, and easy to implement on top of arbitrary off-the-shelf models. We show that this is the first application of symmetrisation to generative modelling, suggesting its potential in this domain more generally.
  arXiv  Detail & Related papers  (2024-10-08T18:02:29Z)
• Relative Representations: Topological and Geometric Perspectives [53.88896255693922]
  Relative representations are an established approach to zero-shot model stitching. We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations. Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• Symplectic Neural Networks Based on Dynamical Systems [0.0]
  We present and analyze a framework for Symplectic neural networks (SympNets) based on geometric for Hamiltonian differential equations. The SympNets are universal approximators in the space of Hamiltonian diffeomorphisms, interpretable and have a non-vanishing property.
  arXiv  Detail & Related papers  (2024-08-19T09:18:28Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• Equivariant neural networks for inverse problems [1.7942265700058986]
  We show that group equivariant convolutional operations can naturally be incorporated into learned reconstruction methods. We design learned iterative methods in which the proximal operators are modelled as group equivariant convolutional neural networks.
  arXiv  Detail & Related papers  (2021-02-23T05:38:41Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.