Accurate Shift Invariant Convolutional Neural Networks Using Gaussian-Hermite Moments

• URL: http://arxiv.org/abs/2603.17098v1
• Date: Tue, 17 Mar 2026 19:37:44 GMT
• Title: Accurate Shift Invariant Convolutional Neural Networks Using Gaussian-Hermite Moments
• Abstract summary: convolutional neural networks (CNNs) are not inherently shift invariant or equivariant.<n>The downsampling operation, used in CNNs, is one of the key reasons which breaks the shift invariant property of a CNN.
• Score: 8.659903550327442
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The convolutional neural networks (CNNs) are not inherently shift invariant or equivariant. The downsampling operation, used in CNNs, is one of the key reasons which breaks the shift invariant property of a CNN. Conversely, downsampling operation is important to improve computational efficiency and increase the area of the receptive field for more contextual information. In this work, we propose Gaussian-Hermite Sampling (GHS), a novel downsampling strategy designed to achieve accurate shift invariance. GHS leverages Gaussian-Hermite polynomials to perform shift-consistent sampling, enabling CNN layers to maintain invariance to arbitrary spatial shifts prior to training. When integrated into standard CNN architectures, the proposed method embeds shift invariance directly at the layer level without requiring architectural modifications or additional training procedures. We evaluate the proposed approach on CIFAR-10, CIFAR-100, and MNIST-rot datasets. Experimental results demonstrate that GHS significantly improves shift consistency, achieving 100% classification consistency under spatial shifts, while also improving classification accuracy compared to baseline CNN models.

Related papers

• Doppler Invariant CNN for Signal Classification [0.0]
  We propose a convolutional neural network (CNN) architecture with complex-valued layers that exploits convolutional shift equivariance in the frequency domain.<n>We show that our model maintains consistent classification accuracy with and without random Doppler shifts despite being trained on no Doppler-shifted examples.
  arXiv  Detail & Related papers  (2025-11-18T16:31:13Z)
• Sorted Convolutional Network for Achieving Continuous Rotational Invariance [56.42518353373004]
  We propose a Sorting Convolution (SC) inspired by some hand-crafted features of texture images. SC achieves continuous rotational invariance without requiring additional learnable parameters or data augmentation. Our results demonstrate that SC achieves the best performance in the aforementioned tasks.
  arXiv  Detail & Related papers  (2023-05-23T18:37:07Z)
• Deep Neural Networks with Efficient Guaranteed Invariances [77.99182201815763]
  We address the problem of improving the performance and in particular the sample complexity of deep neural networks. Group-equivariant convolutions are a popular approach to obtain equivariant representations. We propose a multi-stream architecture, where each stream is invariant to a different transformation.
  arXiv  Detail & Related papers  (2023-03-02T20:44:45Z)
• On the effectiveness of partial variance reduction in federated learning with heterogeneous data [27.527995694042506]
  We show that the diversity of the final classification layers across clients impedes the performance of the FedAvg algorithm. Motivated by this, we propose to correct model by variance reduction only on the final layers. We demonstrate that this significantly outperforms existing benchmarks at a similar or lower communication cost.
  arXiv  Detail & Related papers  (2022-12-05T11:56:35Z)
• Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations [76.82124752950148]
  We develop a convenient gradient-based method for selecting the data augmentation. We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
  arXiv  Detail & Related papers  (2022-02-22T02:51:11Z)
• Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration [77.99182201815763]
  Leveraging prior knowledge on intraclass variance due to transformations is a powerful method to improve the sample complexity of deep neural networks. We propose a novel monomial selection algorithm based on pruning methods to allow an application to more complex problems. We demonstrate the improved sample complexity on the Rotated-MNIST, SVHN and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2022-02-08T16:16:11Z)
• Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks [14.236193187116047]
  Recent studies have put into question the commonly assumed shift invariance property of convolutional networks. We evaluate two methods to improve shift invariance in CNNs, based on low-pass filtering and adaptive sampling of incoming feature maps. We show that these modifications consistently improve sound event classification in all cases considered, without adding any (or adding very few) trainable parameters.
  arXiv  Detail & Related papers  (2021-07-01T17:21:02Z)
• Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling? [59.820507600960745]
  We propose a new GCP meta-layer that uses SVD in the forward pass, and Pad'e Approximants in the backward propagation to compute the gradients. The proposed meta-layer has been integrated into different CNN models and achieves state-of-the-art performances on both large-scale and fine-grained datasets.
  arXiv  Detail & Related papers  (2021-05-06T08:03:45Z)
• Truly shift-invariant convolutional neural networks [0.0]
  Recent works have shown that the output of a CNN can change significantly with small shifts in input. We propose adaptive polyphase sampling (APS), a simple sub-sampling scheme that allows convolutional neural networks to achieve 100% consistency in classification performance under shifts.
  arXiv  Detail & Related papers  (2020-11-28T20:57:35Z)
• Bayesian Graph Neural Networks with Adaptive Connection Sampling [62.51689735630133]
  We propose a unified framework for adaptive connection sampling in graph neural networks (GNNs) The proposed framework not only alleviates over-smoothing and over-fitting tendencies of deep GNNs, but also enables learning with uncertainty in graph analytic tasks with GNNs.
  arXiv  Detail & Related papers  (2020-06-07T07:06:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.