Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency

• URL: http://arxiv.org/abs/2406.07598v4
• Date: Fri, 21 Jun 2024 15:43:36 GMT
• Title: Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency
• Authors: Yuchao Lin, Jacob Helwig, Shurui Gui, Shuiwang Ji,
• Abstract summary: Minimal Frame Averaging (MFA) is a mathematical framework for constructing provably minimal frames that are exactly equivariant. Results demonstrate the efficiency and effectiveness of encoding symmetries via MFA across a diverse range of tasks.
• Score: 48.81897136561015
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider achieving equivariance in machine learning systems via frame averaging. Current frame averaging methods involve a costly sum over large frames or rely on sampling-based approaches that only yield approximate equivariance. Here, we propose Minimal Frame Averaging (MFA), a mathematical framework for constructing provably minimal frames that are exactly equivariant. The general foundations of MFA also allow us to extend frame averaging to more groups than previously considered, including the Lorentz group for describing symmetries in space-time, and the unitary group for complex-valued domains. Results demonstrate the efficiency and effectiveness of encoding symmetries via MFA across a diverse range of tasks, including $n$-body simulation, top tagging in collider physics, and relaxed energy prediction. Our code is available at https://github.com/divelab/MFA.

Related papers

• Batch, match, and patch: low-rank approximations for score-based variational inference [8.840147522046651]
  Black-box variational inference scales poorly to high dimensional problems. We extend the batch-and-match framework for score-based BBVI. We evaluate this approach on a variety of synthetic target distributions and real-world problems in high-dimensional inference.
  arXiv  Detail & Related papers  (2024-10-29T17:42:56Z)
• Incorporating Arbitrary Matrix Group Equivariance into KANs [69.30866522377694]
  We propose Equivariant Kolmogorov-Arnold Networks (EKAN), a method for incorporating arbitrary matrix group equivariants into KANs. EKAN achieves higher accuracy with smaller datasets or fewer parameters on symmetry-related tasks, such as particle scattering and the three-body problem.
  arXiv  Detail & Related papers  (2024-10-01T06:34:58Z)
• Learning Symmetrization for Equivariance with Orbit Distance Minimization [27.284125807569115]
  We present a framework for symmetrizing an arbitrary neural-network architecture and making it equivariant with respect to a given group. We experimentally show our method's competitiveness on the SO(2) image classification task.
  arXiv  Detail & Related papers  (2023-11-13T08:14:29Z)
• Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance [16.49488981364657]
  We present a novel framework to overcome the limitations of equivariant architectures in learning functions with group symmetries. We use an arbitrary base model such as anvariant or a transformer and symmetrize it to be equivariant to the given group. Empirical tests show competitive results against tailored equivariant architectures.
  arXiv  Detail & Related papers  (2023-06-05T13:40:54Z)
• Late Fusion Multi-view Clustering via Global and Local Alignment Maximization [61.89218392703043]
  Multi-view clustering (MVC) optimally integrates complementary information from different views to improve clustering performance. Most of existing approaches directly fuse multiple pre-specified similarities to learn an optimal similarity matrix for clustering. We propose late fusion MVC via alignment to address these issues.
  arXiv  Detail & Related papers  (2022-08-02T01:49:31Z)
• Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation [87.4854250338374]
  We explore relations among affinities in two aspects: single-scale intrinsic correlations and multi-scale relations. Our experiments demonstrate that the proposed method performs favorably against state-of-the-art VSS methods.
  arXiv  Detail & Related papers  (2022-07-21T12:12:36Z)
• Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
  We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. We show that FA-based models have maximal expressive power in a broad setting. We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
  arXiv  Detail & Related papers  (2021-10-07T11:05:23Z)
• PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions [71.60219086238254]
  We deal with the issue from the connection between convolutions and partial differential operators (PDOs) In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs) Experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently.
  arXiv  Detail & Related papers  (2020-07-20T18:57:26Z)
• Sparse Gaussian Processes with Spherical Harmonic Features [14.72311048788194]
  We introduce a new class of inter-domain variational Gaussian processes (GP) Our inference scheme is comparable to variational Fourier features, but it does not suffer from the curse of dimensionality. Our experiments show that our model is able to fit a regression model for a dataset with 6 million entries two orders of magnitude faster.
  arXiv  Detail & Related papers  (2020-06-30T10:19:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.