PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions

• URL: http://arxiv.org/abs/2007.10408v2
• Date: Tue, 11 Aug 2020 14:19:55 GMT
• Title: PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions
• Authors: Zhengyang Shen, Lingshen He, Zhouchen Lin, Jinwen Ma
• Abstract summary: We deal with the issue from the connection between convolutions and partial differential operators (PDOs) In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs) Experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently.
• Score: 71.60219086238254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent research has shown that incorporating equivariance into neural network architectures is very helpful, and there have been some works investigating the equivariance of networks under group actions. However, as digital images and feature maps are on the discrete meshgrid, corresponding equivariance-preserving transformation groups are very limited. In this work, we deal with this issue from the connection between convolutions and partial differential operators (PDOs). In theory, assuming inputs to be smooth, we transform PDOs and propose a system which is equivariant to a much more general continuous group, the $n$-dimension Euclidean group. In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs). Theoretically, the approximation error of PDO-eConvs is of the quadratic order. It is the first time that the error analysis is provided when the equivariance is approximate. Extensive experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently. Particularly, compared with Wide ResNets, our methods result in better results using only 12.6% parameters.

Related papers

• Using and Abusing Equivariance [10.70891251559827]
  We show how Group Equivariant Convolutional Neural Networks use subsampling to learn to break equivariance to their symmetries. We show that a change in the input dimension of a network as small as a single pixel can be enough for commonly used architectures to become approximately equivariant, rather than exactly.
  arXiv  Detail & Related papers  (2023-08-22T09:49:26Z)
• Investigating how ReLU-networks encode symmetries [13.935148870831396]
  We investigate whether equivariance of a network implies that all layers are equivariant. We conjecture that CNNs trained to be equivariant will exhibit layerwise equivariance. We show that it is typically easier to merge a network with a group-transformed version of itself than merging two different networks.
  arXiv  Detail & Related papers  (2023-05-26T15:23:20Z)
• Deep Neural Networks with Efficient Guaranteed Invariances [77.99182201815763]
  We address the problem of improving the performance and in particular the sample complexity of deep neural networks. Group-equivariant convolutions are a popular approach to obtain equivariant representations. We propose a multi-stream architecture, where each stream is invariant to a different transformation.
  arXiv  Detail & Related papers  (2023-03-02T20:44:45Z)
• PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs [69.85869748832127]
  In this work, we employ partial differential operators (PDOs) to model 3D filters, and derive general steerable 3D CNNs called PDO-s3DCNNs. We prove that the equivariant filters are subject to linear constraints, which can be solved efficiently under various conditions.
  arXiv  Detail & Related papers  (2022-08-07T13:37:29Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• Equivariance versus Augmentation for Spherical Images [0.7388859384645262]
  We analyze the role of rotational equivariance in convolutional neural networks (CNNs) applied to spherical images. We compare the performance of the group equivariant networks known as S2CNNs and standard non-equivariant CNNs trained with an increasing amount of data augmentation.
  arXiv  Detail & Related papers  (2022-02-08T16:49:30Z)
• Group Equivariant Subsampling [60.53371517247382]
  Subsampling is used in convolutional neural networks (CNNs) in the form of pooling or strided convolutions. We first introduce translation equivariant subsampling/upsampling layers that can be used to construct exact translation equivariant CNNs. We then generalise these layers beyond translations to general groups, thus proposing group equivariant subsampling/upsampling.
  arXiv  Detail & Related papers  (2021-06-10T16:14:00Z)
• Exploiting Learned Symmetries in Group Equivariant Convolutions [20.63056707649319]
  Group Equivariant Convolutions (GConvs) enable convolutional neural networks to be equivariant to various transformation groups. We show that GConvs can be efficiently decomposed into depthwise separable convolutions.
  arXiv  Detail & Related papers  (2021-06-09T08:50:22Z)
• PDO-e$\text{S}^\text{2}$CNNs: Partial Differential Operator Based Equivariant Spherical CNNs [77.53203546732664]
  We use partial differential operators to design a spherical equivariant CNN, PDO-e$textStext2$CNN, which is exactly rotation equivariant in the continuous domain. In experiments, PDO-e$textStext2$CNNs show greater parameter efficiency and outperform other spherical CNNs significantly on several tasks.
  arXiv  Detail & Related papers  (2021-04-08T07:54:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.