VC dimensions of group convolutional neural networks

• URL: http://arxiv.org/abs/2212.09507v1
• Date: Mon, 19 Dec 2022 14:43:22 GMT
• Title: VC dimensions of group convolutional neural networks
• Authors: Philipp Christian Petersen, Anna Sepliarskaia
• Abstract summary: We study the generalization capacity of group convolutional neural networks. We identify precise estimates for the VC dimensions of simple sets of group convolutional neural networks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the generalization capacity of group convolutional neural networks. We identify precise estimates for the VC dimensions of simple sets of group convolutional neural networks. In particular, we find that for infinite groups and appropriately chosen convolutional kernels, already two-parameter families of convolutional neural networks have an infinite VC dimension, despite being invariant to the action of an infinite group.

Related papers

• On the VC dimension of deep group convolutional neural networks [7.237068561453083]
  We study the generalization capabilities of Group Convolutional Neural Networks (GCNNs) with ReLU activation function. We analyze how factors such as the number of layers, weights, and input dimension affect the Vapnik-Chervonenkis (VC) dimension.
  arXiv  Detail & Related papers  (2024-10-21T09:16:06Z)
• Equivariant Neural Tangent Kernels [2.373992571236766]
  We give explicit expressions for neural tangent kernels (NTKs) of group convolutional neural networks. In numerical experiments, we demonstrate superior performance for equivariant NTKs over non-equivariant NTKs on a classification task for medical images.
  arXiv  Detail & Related papers  (2024-06-10T17:43:13Z)
• Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime [52.00917519626559]
  This paper presents two models of neural-networks and their training applicable to neural networks of arbitrary width, depth and topology. We also present an exact novel representor theory for layer-wise neural network training with unregularized gradient descent in terms of a local-extrinsic neural kernel (LeNK) This representor theory gives insight into the role of higher-order statistics in neural network training and the effect of kernel evolution in neural-network kernel models.
  arXiv  Detail & Related papers  (2024-05-24T06:30:36Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• Norm-based Generalization Bounds for Compositionally Sparse Neural Networks [11.987589603961622]
  We prove generalization bounds for multilayered sparse ReLU neural networks, including convolutional neural networks. Taken together, these results suggest that compositional sparsity of the underlying target function is critical to the success of deep neural networks.
  arXiv  Detail & Related papers  (2023-01-28T00:06:22Z)
• Equivariant and Steerable Neural Networks: A review with special emphasis on the symmetric group [0.76146285961466]
  Convolutional neural networks revolutionized computer vision and natrual language processing. Their efficiency, as compared to fully connected neural networks, has its origin in the architecture. We review the architecture of such networks including equivariant layers and filter banks, activation with capsules and group pooling.
  arXiv  Detail & Related papers  (2023-01-08T11:05:31Z)
• Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK [86.45209429863858]
  We study training one-hidden-layer ReLU networks in the neural tangent kernel (NTK) regime. We show that the neural networks possess a different limiting kernel which we call textitbias-generalized NTK We also study various properties of the neural networks with this new kernel.
  arXiv  Detail & Related papers  (2023-01-01T02:11:39Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Identifying and interpreting tuning dimensions in deep networks [83.59965686504822]
  tuning dimension is a stimulus attribute that accounts for much of the activation variance of a group of neurons. This work contributes an unsupervised framework for identifying and interpreting "tuning dimensions" in deep networks.
  arXiv  Detail & Related papers  (2020-11-05T21:26:03Z)
• Neural Group Actions [0.0]
  We introduce an algorithm for designing Neural Group Actions, collections of deep neural network architectures which model symmetric transformations satisfying the laws of a given finite group. We demonstrate experimentally that a Neural Group Action for the quaternion group $Q_8$ can learn how a set of nonuniversal quantum gates satisfying the $Q_8$ group laws act on single qubit quantum states.
  arXiv  Detail & Related papers  (2020-10-08T02:27:05Z)
• Learn to Predict Sets Using Feed-Forward Neural Networks [63.91494644881925]
  This paper addresses the task of set prediction using deep feed-forward neural networks. We present a novel approach for learning to predict sets with unknown permutation and cardinality. We demonstrate the validity of our set formulations on relevant vision problems.
  arXiv  Detail & Related papers  (2020-01-30T01:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.