Expressive Sign Equivariant Networks for Spectral Geometric Learning

• URL: http://arxiv.org/abs/2312.02339v1
• Date: Mon, 4 Dec 2023 20:48:18 GMT
• Title: Expressive Sign Equivariant Networks for Spectral Geometric Learning
• Authors: Derek Lim and Joshua Robinson and Stefanie Jegelka and Haggai Maron
• Abstract summary: Recent work has shown the utility of developing machine learning models that respect the structure and symmetries of eigenvectors. We show that sign invariance is theoretically limited for tasks such as building equivariant models and learning node positional encodings for link prediction in graphs.
• Score: 47.71042325868781
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work has shown the utility of developing machine learning models that respect the structure and symmetries of eigenvectors. These works promote sign invariance, since for any eigenvector v the negation -v is also an eigenvector. However, we show that sign invariance is theoretically limited for tasks such as building orthogonally equivariant models and learning node positional encodings for link prediction in graphs. In this work, we demonstrate the benefits of sign equivariance for these tasks. To obtain these benefits, we develop novel sign equivariant neural network architectures. Our models are based on a new analytic characterization of sign equivariant polynomials and thus inherit provable expressiveness properties. Controlled synthetic experiments show that our networks can achieve the theoretically predicted benefits of sign equivariant models. Code is available at https://github.com/cptq/Sign-Equivariant-Nets.

Related papers

• A Characterization Theorem for Equivariant Networks with Point-wise Activations [13.00676132572457]
  We prove that rotation-equivariant networks can only be invariant, as it happens for any network which is equivariant with respect to connected compact groups. We show that feature spaces of disentangled steerable convolutional neural networks are trivial representations.
  arXiv  Detail & Related papers  (2024-01-17T14:30:46Z)
• Symmetry Breaking and Equivariant Neural Networks [17.740760773905986]
  We introduce a novel notion of'relaxed equiinjection' We show how to incorporate this relaxation into equivariant multilayer perceptronrons (E-MLPs) The relevance of symmetry breaking is then discussed in various application domains.
  arXiv  Detail & Related papers  (2023-12-14T15:06:48Z)
• Non Commutative Convolutional Signal Models in Neural Networks: Stability to Small Deformations [111.27636893711055]
  We study the filtering and stability properties of non commutative convolutional filters. Our results have direct implications for group neural networks, multigraph neural networks and quaternion neural networks.
  arXiv  Detail & Related papers  (2023-10-05T20:27:22Z)
• The Lie Derivative for Measuring Learned Equivariance [84.29366874540217]
  We study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. We find that many violations of equivariance can be linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities. For example, transformers can be more equivariant than convolutional neural networks after training.
  arXiv  Detail & Related papers  (2022-10-06T15:20:55Z)
• Sign and Basis Invariant Networks for Spectral Graph Representation Learning [75.18802152811539]
  We introduce SignNet and BasisNet -- new neural architectures that are invariant to all requisite symmetries and hence process collections of eigenspaces in a principled manner. Our networks are theoretically strong for graph representation learning -- they can approximate any spectral graph convolution. Experiments show the strength of our networks for learning spectral graph filters and learning graph positional encodings.
  arXiv  Detail & Related papers  (2022-02-25T23:11:59Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
  We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. We show that FA-based models have maximal expressive power in a broad setting. We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
  arXiv  Detail & Related papers  (2021-10-07T11:05:23Z)
• Disentangled Representation Learning and Generation with Manifold Optimization [10.69910379275607]
  This work presents a representation learning framework that explicitly promotes disentanglement by encouraging directions of variations. Our theoretical discussion and various experiments show that the proposed model improves over many VAE variants in terms of both generation quality and disentangled representation learning.
  arXiv  Detail & Related papers  (2020-06-12T10:00:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.