Stochastic Neural Network Symmetrisation in Markov Categories

• URL: http://arxiv.org/abs/2406.11814v1
• Date: Mon, 17 Jun 2024 17:54:42 GMT
• Title: Stochastic Neural Network Symmetrisation in Markov Categories
• Authors: Rob Cornish,
• Abstract summary: We consider the problem of symmetrising a neural network along a group homomorphism. We obtain a flexible, compositional, and generic framework for symmetrisation.
• Score: 2.0668277618112203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of symmetrising a neural network along a group homomorphism: given a homomorphism $\varphi : H \to G$, we would like a procedure that converts $H$-equivariant neural networks into $G$-equivariant ones. We formulate this in terms of Markov categories, which allows us to consider neural networks whose outputs may be stochastic, but with measure-theoretic details abstracted away. We obtain a flexible, compositional, and generic framework for symmetrisation that relies on minimal assumptions about the structure of the group and the underlying neural network architecture. Our approach recovers existing methods for deterministic symmetrisation as special cases, and extends directly to provide a novel methodology for stochastic symmetrisation also. Beyond this, we believe our findings also demonstrate the utility of Markov categories for addressing problems in machine learning in a conceptual yet mathematically rigorous way.

Related papers

• Group Crosscoders for Mechanistic Analysis of Symmetry [0.0]
  Group crosscoders systematically discover and analyse symmetrical features in neural networks. We show that group crosscoders can provide systematic insights into how neural networks represent symmetry.
  arXiv  Detail & Related papers  (2024-10-31T17:47:01Z)
• SymDiff: Equivariant Diffusion via Stochastic Symmetrisation [28.614292092399563]
  We propose a novel method for constructing equivariant diffusion models using the recently introduced framework of symmetrisation. SymDiff resembles a learned data augmentation that is deployed at sampling time, and is lightweight, computationally efficient, and easy to implement on top of arbitrary off-the-shelf models. We show that this is the first application of symmetrisation to generative modelling, suggesting its potential in this domain more generally.
  arXiv  Detail & Related papers  (2024-10-08T18:02:29Z)
• Relative Representations: Topological and Geometric Perspectives [53.88896255693922]
  Relative representations are an established approach to zero-shot model stitching. We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations. Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof [50.49582712378289]
  We investigate the impact of neural parameter symmetries by introducing new neural network architectures. We develop two methods, with some provable guarantees, of modifying standard neural networks to reduce parameter space symmetries. Our experiments reveal several interesting observations on the empirical impact of parameter symmetries.
  arXiv  Detail & Related papers  (2024-05-30T16:32:31Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Symmetry Breaking and Equivariant Neural Networks [17.740760773905986]
  We introduce a novel notion of'relaxed equiinjection' We show how to incorporate this relaxation into equivariant multilayer perceptronrons (E-MLPs) The relevance of symmetry breaking is then discussed in various application domains.
  arXiv  Detail & Related papers  (2023-12-14T15:06:48Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• Equivariant neural networks for inverse problems [1.7942265700058986]
  We show that group equivariant convolutional operations can naturally be incorporated into learned reconstruction methods. We design learned iterative methods in which the proximal operators are modelled as group equivariant convolutional neural networks.
  arXiv  Detail & Related papers  (2021-02-23T05:38:41Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Detecting Symmetries with Neural Networks [0.0]
  We make extensive use of the structure in the embedding layer of the neural network. We identify whether a symmetry is present and to identify orbits of the symmetry in the input. For this example we present a novel data representation in terms of graphs.
  arXiv  Detail & Related papers  (2020-03-30T17:58:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.