Symmetries in PAC-Bayesian Learning

• URL: http://arxiv.org/abs/2510.17303v1
• Date: Mon, 20 Oct 2025 08:45:57 GMT
• Title: Symmetries in PAC-Bayesian Learning
• Authors: Armin Beck, Peter Ochs,
• Abstract summary: We extend generalization guarantees to the broader setting of non-compact symmetries.<n>We validate our theory with experiments on a rotated MNIST dataset with a non-uniform rotation group.
• Score: 0.9023847175654601
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Symmetries are known to improve the empirical performance of machine learning models, yet theoretical guarantees explaining these gains remain limited. Prior work has focused mainly on compact group symmetries and often assumes that the data distribution itself is invariant, an assumption rarely satisfied in real-world applications. In this work, we extend generalization guarantees to the broader setting of non-compact symmetries, such as translations and to non-invariant data distributions. Building on the PAC-Bayes framework, we adapt and tighten existing bounds, demonstrating the approach on McAllester's PAC-Bayes bound while showing that it applies to a wide range of PAC-Bayes bounds. We validate our theory with experiments on a rotated MNIST dataset with a non-uniform rotation group, where the derived guarantees not only hold but also improve upon prior results. These findings provide theoretical evidence that, for symmetric data, symmetric models are preferable beyond the narrow setting of compact groups and invariant distributions, opening the way to a more general understanding of symmetries in machine learning.

Related papers

• Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation [56.361076943802594]
  CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.
  arXiv  Detail & Related papers  (2026-02-16T18:58:55Z)
• Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limits [5.63508094975827]
  Equivariant diffusion policies (EDPs) combine the generative expressivity of diffusion models with the strong generalization and sample efficiency afforded by geometric symmetries.<n>We show that exploiting symmetry during the steering process yields substantial benefits-enhancing sample efficiency, preventing value divergence, and achieving strong policy improvements even when EDPs are trained from extremely limited demonstrations.
  arXiv  Detail & Related papers  (2025-12-12T07:42:01Z)
• Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments [10.122552307413711]
  Group symmetries provide a powerful inductive bias for reinforcement learning (RL)<n>Group symmetries provide a powerful inductive bias for reinforcement learning (RL)
  arXiv  Detail & Related papers  (2025-11-30T14:41:08Z)
• Reinforcement Learning Using known Invariances [54.91261509214309]
  This paper develops a theoretical framework for incorporating known group symmetries into kernel-based reinforcement learning.<n>We show that symmetry-aware RL achieves significantly better performance than their standard kernel counterparts.
  arXiv  Detail & Related papers  (2025-11-05T13:56:14Z)
• A Regularization-Guided Equivariant Approach for Image Restoration [46.44312175792672]
  Equivariant and invariant deep learning models have been developed to exploit intrinsic symmetries in data.<n>These methods often suffer from limited representation accuracy and rely on strict symmetry assumptions that may not hold in practice.<n>We propose a rotation-equivariant regularization strategy that adaptively enforces the appropriate symmetry constraints on the data while preserving the network's representational accuracy.
  arXiv  Detail & Related papers  (2025-05-26T10:30:26Z)
• Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors [46.59269589647962]
  Group equivariance has emerged as a valuable inductive bias in deep learning.<n>Group equivariant methods require the groups of interest to be known beforehand.<n>We show that when the dataset exhibits strong symmetries, the permutation matrices will converge to regular group representations.
  arXiv  Detail & Related papers  (2024-12-05T20:15:34Z)
• Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups [11.572188414440436]
  We propose Lie aLgebrA Canonicalization (LieLAC), a novel approach that exploits only the action of infinitesimal generators of the symmetry group.<n> operating within the framework of canonicalization, LieLAC can easily be integrated with unconstrained pre-trained models.
  arXiv  Detail & Related papers  (2024-10-03T17:21:30Z)
• Uniform Transformation: Refining Latent Representation in Variational Autoencoders [7.4316292428754105]
  We introduce a novel adaptable three-stage Uniform Transformation (UT) module to address irregular latent distributions. By reconfiguring irregular distributions into a uniform distribution in the latent space, our approach significantly enhances the disentanglement and interpretability of latent representations. Empirical evaluations demonstrated the efficacy of our proposed UT module in improving disentanglement metrics across benchmark datasets.
  arXiv  Detail & Related papers  (2024-07-02T21:46:23Z)
• SymmPI: Predictive Inference for Data with Group Symmetries [20.772826042110633]
  We propose SymmPI, a methodology for predictive inference when data distributions have general group symmetries. Our methods leverage the novel notion of distributional equivariant transformations. We show that SymmPI has valid coverage under distributional invariance and characterize its performance under distribution shift.
  arXiv  Detail & Related papers  (2023-12-26T18:41:14Z)
• Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computation [52.73824786627612]
  This paper establishes new convergence results for textitgeodesic strongly monotone games.<n>Our key result shows that RGD attains last-iterate linear convergence in a textitgeometry-agnostic fashion.<n>Overall, this paper presents the first geometry-agnostic last-iterate convergence analysis for games beyond the Euclidean settings.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Wrapped Distributions on homogeneous Riemannian manifolds [58.720142291102135]
  Control over distributions' properties, such as parameters, symmetry and modality yield a family of flexible distributions. We empirically validate our approach by utilizing our proposed distributions within a variational autoencoder and a latent space network model.
  arXiv  Detail & Related papers  (2022-04-20T21:25:21Z)
• Non-Linear Spectral Dimensionality Reduction Under Uncertainty [107.01839211235583]
  We propose a new dimensionality reduction framework, called NGEU, which leverages uncertainty information and directly extends several traditional approaches. We show that the proposed NGEU formulation exhibits a global closed-form solution, and we analyze, based on the Rademacher complexity, how the underlying uncertainties theoretically affect the generalization ability of the framework.
  arXiv  Detail & Related papers  (2022-02-09T19:01:33Z)
• Equivariant Manifold Flows [48.21296508399746]
  We lay the theoretical foundations for learning symmetry-invariant distributions on arbitrary manifold via equivariant manifold flows. We demonstrate the utility of our approach by using it to learn gauge invariant densities over $SU(n)$ in the context of quantum field theory.
  arXiv  Detail & Related papers  (2021-07-19T03:04:44Z)
• Von Mises-Fisher Elliptical Distribution [5.7559253770425425]
  We propose to employ the von-Mises-Fisher (vMF) distribution to obtain an explicit and simple probability representation of a skewed elliptical distribution. This is shown not only to allow us to deal with non-symmetric learning systems, but also to provide a physically meaningful way of generalising skewed distributions. We also demonstrate that the proposed vMF distribution is both easy to generate and stable to estimate, both theoretically and through examples.
  arXiv  Detail & Related papers  (2021-03-14T15:14:04Z)
• Generalized Sliced Distances for Probability Distributions [47.543990188697734]
  We introduce a broad family of probability metrics, coined as Generalized Sliced Probability Metrics (GSPMs) GSPMs are rooted in the generalized Radon transform and come with a unique geometric interpretation. We consider GSPM-based gradient flows for generative modeling applications and show that under mild assumptions, the gradient flow converges to the global optimum.
  arXiv  Detail & Related papers  (2020-02-28T04:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.