Parameter-Efficient Architectural Modifications for Translation-Invariant CNNs

• URL: http://arxiv.org/abs/2604.27870v1
• Date: Thu, 30 Apr 2026 13:52:42 GMT
• Title: Parameter-Efficient Architectural Modifications for Translation-Invariant CNNs
• Abstract summary: Convolutional Neural Networks (CNNs) are widely assumed to be translation-invariant, yet standard architectures exhibit a startling fragility.<n>We propose a lightweight 'Online Architecture' strategy by strategically inserting Global Average Pooling layers at various network depths.<n>This strategy achieves a massive 98% reduction in trainable parameters (from 5.2M to just 82K) and a 90% reduction in total network size.
• Score: 2.111646739533638
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolutional Neural Networks (CNNs) are widely assumed to be translation-invariant, yet standard architectures exhibit a startling fragility: even a single-pixel shift can drastically degrade performance due to their reliance on spatially dependent fully connected layers. In this work, we resolve this vulnerability by proposing a lightweight 'Online Architecture' strategy. By strategically inserting Global Average Pooling (GAP) layers at various network depths, we effectively decouple feature recognition from spatial location. Using VGG-16 as a primary case study, we demonstrate that this architectural modification achieves a massive 98% reduction in trainable parameters (from 5.2M to just 82K) and a 90% reduction in total network size (138M to 14M). Despite this drastic pruning, our variants maintain competitive Top-1 accuracy on ImageNet (66.4%) while doubling translational robustness, reducing average relative loss from 0.09 to 0.05. Furthermore, our analysis identifies a fundamental limit to invariance: while GAP resolves macroscopic sensitivity, discrete pooling operations introduce a residual periodic aliasing that prevents perfect pixel-level stability. Finally, we extend these findings to Perceptual Image Quality Assessment (IQA) by integrating our invariant backbones into the LPIPS framework. The resulting metric significantly outperforms the retrained baseline in generalization across the KADID-10k dataset (Spearman 0.89 vs. 0.75) and achieves a near-perfect alignment with human psychophysical response curves on the RAID dataset (Spearman 0.95). These results confirm that enforcing architectural invariance is a far more efficient and biologically plausible path to robustness than traditional data augmentation. Data and code are publicly available. The data and code are publicly available to facilitate validation and further research.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• When Less Is More: A Controlled Benchmark of Lightweight CNNs for Satellite Land-Cover Segmentation on DeepGlobe [0.3093890460224435]
  This study compares five architectures VGG16, MobileNetV2, InceptionV3, AlexNet, and CNN on the DeepGlobe Land Cover Classification dataset.<n>At 24.98 MB, MobileNetV2_v1 had the highest overall accuracy (0.7906) and mean Intersection over Union (0.4625)<n>These results show that lightweight, transfer-learned models can match or outperform deeper models in resource-constrained remote-sensing environments.
  arXiv  Detail & Related papers  (2026-07-25T03:51:49Z)
• Efficient Hybrid CNN-GNN Architecture for Monocular Depth Estimation [0.0]
  We present GraphDepth, a monocular depth estimation architecture that integrates Graph Neural Networks (GNNs) within a convolutional encoder-decoder framework.<n>Our approach embeds efficient GraphSAGE layers at multiple scales of a ResNet-101 U-Net backbone.<n> Experiments on NYU Depth V2, WHU Aerial, ETH3D, and Mid-Air benchmarks demonstrate competitive accuracy within 4.6% of state-of-the-art transformers.
  arXiv  Detail & Related papers  (2026-05-11T09:21:04Z)
• Alternating Gradient Flow Utility: A Unified Metric for Structural Pruning and Dynamic Routing in Deep Networks [52.153950303594684]
  We propose a decoupled kinetic paradigm inspired by Alternating Gradient Flow (AGF)<n>AGF accurately captures the network's structural "kinetic utility"<n>We design a hybrid routing framework that decouples AGF-guided offline structural search from online execution via zero-cost physical priors.
  arXiv  Detail & Related papers  (2026-03-12T18:19:21Z)
• On the Relationship Between Representation Geometry and Generalization in Deep Neural Networks [0.0]
  We show that effective dimension -- an unsupervised geometric metric -- strongly predicts accuracy.<n>We establish causality: degrading geometry via noise causes accuracy loss (r=-0.94, $p 10(9)$), while improving geometry via PCA maintains accuracy across architectures (-0.03pp at 95% variance)<n>These results establish that effective dimension provides domain-agnostic predictive and causal information about neural network performance, computed entirely without labels.
  arXiv  Detail & Related papers  (2026-01-28T04:33:41Z)
• Filter Pruning For CNN With Enhanced Linear Representation Redundancy [3.853146967741941]
  We present a data-driven loss function term calculated from the correlation matrix of different feature maps in the same layer, named CCM-loss. CCM-loss provides us with another universal transcendental mathematical tool besides L*-norm regularization. In our new strategy, we mainly focus on the consistency and integrality of the information flow in the network.
  arXiv  Detail & Related papers  (2023-10-10T06:27:30Z)
• Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks [48.089501687522954]
  We propose a novel layer-adaptive weight-pruning approach for Deep Neural Networks (DNNs) Our approach takes into account the collective influence of all layers to design a layer-adaptive pruning scheme. Our experiments demonstrate the superiority of our approach over existing methods on the ImageNet and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2023-08-21T03:22:47Z)
• One-Shot Learning for Periocular Recognition: Exploring the Effect of Domain Adaptation and Data Bias on Deep Representations [59.17685450892182]
  We investigate the behavior of deep representations in widely used CNN models under extreme data scarcity for One-Shot periocular recognition. We improved state-of-the-art results that made use of networks trained with biometric datasets with millions of images. Traditional algorithms like SIFT can outperform CNNs in situations with limited data.
  arXiv  Detail & Related papers  (2023-07-11T09:10:16Z)
• SmoothNets: Optimizing CNN architecture design for differentially private deep learning [69.10072367807095]
  DPSGD requires clipping and noising of per-sample gradients. This introduces a reduction in model utility compared to non-private training. We distilled a new model architecture termed SmoothNet, which is characterised by increased robustness to the challenges of DP-SGD training.
  arXiv  Detail & Related papers  (2022-05-09T07:51:54Z)
• Global Vision Transformer Pruning with Hessian-Aware Saliency [93.33895899995224]
  This work challenges the common design philosophy of the Vision Transformer (ViT) model with uniform dimension across all the stacked blocks in a model stage. We derive a novel Hessian-based structural pruning criteria comparable across all layers and structures, with latency-aware regularization for direct latency reduction. Performing iterative pruning on the DeiT-Base model leads to a new architecture family called NViT (Novel ViT), with a novel parameter that utilizes parameters more efficiently.
  arXiv  Detail & Related papers  (2021-10-10T18:04:59Z)
• Semantic Perturbations with Normalizing Flows for Improved Generalization [62.998818375912506]
  We show that perturbations in the latent space can be used to define fully unsupervised data augmentations. We find that our latent adversarial perturbations adaptive to the classifier throughout its training are most effective.
  arXiv  Detail & Related papers  (2021-08-18T03:20:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.