Frequency Pooling: Shift-Equivalent and Anti-Aliasing Downsampling

• URL: http://arxiv.org/abs/2109.11839v1
• Date: Fri, 24 Sep 2021 09:32:10 GMT
• Title: Frequency Pooling: Shift-Equivalent and Anti-Aliasing Downsampling
• Authors: Zhendong Zhang
• Abstract summary: We show that frequency pooling is shift-equivalent and anti-aliasing based on the property of Fourier transform and Nyquist frequency. Experiments on image classification show that frequency pooling improves accuracy and robustness with respect to the shifts of CNNs.
• Score: 9.249235534786072
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolution utilizes a shift-equivalent prior of images, thus leading to great success in image processing tasks. However, commonly used poolings in convolutional neural networks (CNNs), such as max-pooling, average-pooling, and strided-convolution, are not shift-equivalent. Thus, the shift-equivalence of CNNs is destroyed when convolutions and poolings are stacked. Moreover, anti-aliasing is another essential property of poolings from the perspective of signal processing. However, recent poolings are neither shift-equivalent nor anti-aliasing. To address this issue, we propose a new pooling method that is shift-equivalent and anti-aliasing, named frequency pooling. Frequency pooling first transforms the features into the frequency domain, and then removes the frequency components beyond the Nyquist frequency. Finally, it transforms the features back to the spatial domain. We prove that frequency pooling is shift-equivalent and anti-aliasing based on the property of Fourier transform and Nyquist frequency. Experiments on image classification show that frequency pooling improves accuracy and robustness with respect to the shifts of CNNs.

Related papers

• FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization [67.47895278233717]
  We develop a progressive frequency regularization technique to tackle the over-reconstruction issue within the frequency space. FreGS achieves superior novel view synthesis and outperforms the state-of-the-art consistently.
  arXiv  Detail & Related papers  (2024-03-11T17:00:27Z)
• Misalignment-Robust Frequency Distribution Loss for Image Transformation [51.0462138717502]
  This paper aims to address a common challenge in deep learning-based image transformation methods, such as image enhancement and super-resolution. We introduce a novel and simple Frequency Distribution Loss (FDL) for computing distribution distance within the frequency domain. Our method is empirically proven effective as a training constraint due to the thoughtful utilization of global information in the frequency domain.
  arXiv  Detail & Related papers  (2024-02-28T09:27:41Z)
• Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling [11.72025865314187]
  Convolutional neural networks encode images through a sequence of convolutions, normalizations and non-linearities as well as downsampling operations. Previous work showed that even slight mistakes during sampling, leading to aliasing, can be directly attributed to the networks' lack in robustness. We propose aliasing and spectral artifact-free pooling, short ASAP.
  arXiv  Detail & Related papers  (2023-07-19T07:47:23Z)
• On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks [0.0]
  Subsampled convolutions with Gabor-like filters are prone to aliasing, causing sensitivity to small input shifts. We highlight the crucial role played by the filter's frequency and orientation in achieving stability. We experimentally validate our theory by considering a deterministic feature extractor based on the dual-tree complex wavelet packet transform.
  arXiv  Detail & Related papers  (2022-09-19T08:15:30Z)
• Hierarchical Spherical CNNs with Lifting-based Adaptive Wavelets for Pooling and Unpooling [101.72318949104627]
  We propose a novel framework of hierarchical convolutional neural networks (HS-CNNs) with a lifting structure to learn adaptive spherical wavelets for pooling and unpooling. LiftHS-CNN ensures a more efficient hierarchical feature learning for both image- and pixel-level tasks.
  arXiv  Detail & Related papers  (2022-05-31T07:23:42Z)
• Adaptive Frequency Learning in Two-branch Face Forgery Detection [66.91715092251258]
  We propose Adaptively learn Frequency information in the two-branch Detection framework, dubbed AFD. We liberate our network from the fixed frequency transforms, and achieve better performance with our data- and task-dependent transform layers.
  arXiv  Detail & Related papers  (2022-03-27T14:25:52Z)
• Exploring Novel Pooling Strategies for Edge Preserved Feature Maps in Convolutional Neural Networks [0.0]
  Anti-aliased convolutional neural networks (CNNs) have led to some resurgence in relooking the way pooling is done in CNNs. Two novel pooling approaches are presented such as Laplacian-Gaussian Concatenation with Attention (LGCA) pooling and Wavelet based approximate-detailed concatenation with attention (WADCA) pooling. Results suggest that the proposed pooling approaches outperform the conventional pooling as well as blur pooling for classification, segmentation and autoencoders.
  arXiv  Detail & Related papers  (2021-10-17T15:11:51Z)
• Dense Pruning of Pointwise Convolutions in the Frequency Domain [10.58456555092086]
  We propose a technique which wraps each pointwise layer in a discrete cosine transform (DCT) which is truncated to selectively prune coefficients above a given threshold. Unlike weight pruning techniques which rely on sparse operators, our contiguous frequency band pruning results in fully dense computation. We apply our technique to MobileNetV2 and in the process reduce computation time by 22% and incur 1% accuracy degradation.
  arXiv  Detail & Related papers  (2021-09-16T04:02:45Z)
• WaveFill: A Wavelet-based Generation Network for Image Inpainting [57.012173791320855]
  WaveFill is a wavelet-based inpainting network that decomposes images into multiple frequency bands. WaveFill decomposes images by using discrete wavelet transform (DWT) that preserves spatial information naturally. It applies L1 reconstruction loss to the low-frequency bands and adversarial loss to high-frequency bands, hence effectively mitigate inter-frequency conflicts.
  arXiv  Detail & Related papers  (2021-07-23T04:44:40Z)
• WaveTransform: Crafting Adversarial Examples via Input Decomposition [69.01794414018603]
  We introduce WaveTransform', that creates adversarial noise corresponding to low-frequency and high-frequency subbands, separately (or in combination) Experiments show that the proposed attack is effective against the defense algorithm and is also transferable across CNNs.
  arXiv  Detail & Related papers  (2020-10-29T17:16:59Z)
• Robust Learning with Frequency Domain Regularization [1.370633147306388]
  We introduce a new regularization method by constraining the frequency spectra of the filter of the model. We demonstrate the effectiveness of our regularization by (1) defensing to adversarial perturbations; (2) reducing the generalization gap in different architecture; and (3) improving the generalization ability in transfer learning scenario without fine-tune.
  arXiv  Detail & Related papers  (2020-07-07T07:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.