Optimal Symmetries in Binary Classification

• URL: http://arxiv.org/abs/2408.08823v1
• Date: Fri, 16 Aug 2024 16:15:18 GMT
• Title: Optimal Symmetries in Binary Classification
• Authors: Vishal S. Ngairangbam, Michael Spannowsky,
• Abstract summary: We show that selecting the appropriate group symmetries is crucial for optimising generalisation and sample efficiency. We develop a theoretical foundation for designing group equivariant neural networks that align the choice of symmetries with the underlying probability distributions of the data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore the role of group symmetries in binary classification tasks, presenting a novel framework that leverages the principles of Neyman-Pearson optimality. Contrary to the common intuition that larger symmetry groups lead to improved classification performance, our findings show that selecting the appropriate group symmetries is crucial for optimising generalisation and sample efficiency. We develop a theoretical foundation for designing group equivariant neural networks that align the choice of symmetries with the underlying probability distributions of the data. Our approach provides a unified methodology for improving classification accuracy across a broad range of applications by carefully tailoring the symmetry group to the specific characteristics of the problem. Theoretical analysis and experimental results demonstrate that optimal classification performance is not always associated with the largest equivariant groups possible in the domain, even when the likelihood ratio is invariant under one of its proper subgroups, but rather with those subgroups themselves. This work offers insights and practical guidelines for constructing more effective group equivariant architectures in diverse machine-learning contexts.

Related papers

• Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors [46.59269589647962]
  Group equivariance has emerged as a valuable inductive bias in deep learning. Group equivariant methods require the groups of interest to be known beforehand. We show that when the dataset exhibits strong symmetries, the permutation matrices will converge to regular group representations.
  arXiv  Detail & Related papers  (2024-12-05T20:15:34Z)
• Adaptive Transfer Clustering: A Unified Framework [2.3144964550307496]
  We propose an adaptive transfer clustering (ATC) algorithm that automatically leverages the commonality in the presence of unknown discrepancy. It applies to a broad class of statistical models including Gaussian mixture models, block models, and latent class models.
  arXiv  Detail & Related papers  (2024-10-28T17:57:06Z)
• Symmetry Discovery for Different Data Types [52.2614860099811]
  Equivariant neural networks incorporate symmetries into their architecture, achieving higher generalization performance. We propose LieSD, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks. We validate the performance of LieSD on tasks with symmetries such as the two-body problem, the moment of inertia matrix prediction, and top quark tagging.
  arXiv  Detail & Related papers  (2024-10-13T13:39:39Z)
• Equivariant score-based generative models provably learn distributions with symmetries efficiently [7.90752151686317]
  Empirical studies have demonstrated that incorporating symmetries into generative models can provide better generalization and sampling efficiency. We provide the first theoretical analysis and guarantees of score-based generative models (SGMs) for learning distributions that are invariant with respect to some group symmetry.
  arXiv  Detail & Related papers  (2024-10-02T05:14:28Z)
• Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification [72.77513633290056]
  We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model. Our method captures intricate patterns and relationships, enhancing classification performance.
  arXiv  Detail & Related papers  (2024-02-14T16:10:42Z)
• Architectural Optimization over Subgroups for Equivariant Neural Networks [0.0]
  We propose equivariance relaxation morphism and $[G]$-mixed equivariant layer to operate with equivariance constraints on a subgroup. We present evolutionary and differentiable neural architecture search (NAS) algorithms that utilize these mechanisms respectively for equivariance-aware architectural optimization.
  arXiv  Detail & Related papers  (2022-10-11T14:37:29Z)
• Universal Approximation Theorem for Equivariant Maps by Group CNNs [14.810452619505137]
  This paper provides a unified method to obtain universal approximation theorems for equivariant maps by CNNs. As its significant advantage, we can handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.
  arXiv  Detail & Related papers  (2020-12-27T07:09:06Z)
• LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
  Group equivariant neural networks are used as building blocks of group invariant neural networks. We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
  arXiv  Detail & Related papers  (2020-12-20T11:02:49Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)
• Stochastic Flows and Geometric Optimization on the Orthogonal Group [52.50121190744979]
  We present a new class of geometrically-driven optimization algorithms on the orthogonal group $O(d)$. We show that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, flows and metric learning.
  arXiv  Detail & Related papers  (2020-03-30T15:37:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.