Truly Scale-Equivariant Deep Nets with Fourier Layers

• URL: http://arxiv.org/abs/2311.02922v1
• Date: Mon, 6 Nov 2023 07:32:27 GMT
• Title: Truly Scale-Equivariant Deep Nets with Fourier Layers
• Authors: Md Ashiqur Rahman, Raymond A. Yeh
• Abstract summary: In computer vision, models must be able to adapt to changes in image resolution to effectively carry out tasks such as image segmentation. Recent works have made progress in developing scale-equivariant convolutional neural networks, through weight-sharing and kernel resizing. We propose a novel architecture based on Fourier layers to achieve truly scale-equivariant deep nets.
• Score: 14.072558848402362
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In computer vision, models must be able to adapt to changes in image resolution to effectively carry out tasks such as image segmentation; This is known as scale-equivariance. Recent works have made progress in developing scale-equivariant convolutional neural networks, e.g., through weight-sharing and kernel resizing. However, these networks are not truly scale-equivariant in practice. Specifically, they do not consider anti-aliasing as they formulate the down-scaling operation in the continuous domain. To address this shortcoming, we directly formulate down-scaling in the discrete domain with consideration of anti-aliasing. We then propose a novel architecture based on Fourier layers to achieve truly scale-equivariant deep nets, i.e., absolute zero equivariance-error. Following prior works, we test this model on MNIST-scale and STL-10 datasets. Our proposed model achieves competitive classification performance while maintaining zero equivariance-error.

Related papers

• Tilt your Head: Activating the Hidden Spatial-Invariance of Classifiers [0.7704032792820767]
  Deep neural networks are applied in more and more areas of everyday life. They still lack essential abilities, such as robustly dealing with spatially transformed input signals. We propose a novel technique to emulate such an inference process for neural nets.
  arXiv  Detail & Related papers  (2024-05-06T09:47:29Z)
• Neural Fields with Thermal Activations for Arbitrary-Scale Super-Resolution [56.089473862929886]
  We present a novel way to design neural fields such that points can be queried with an adaptive Gaussian PSF. With its theoretically guaranteed anti-aliasing, our method sets a new state of the art for arbitrary-scale single image super-resolution.
  arXiv  Detail & Related papers  (2023-11-29T14:01:28Z)
• Revisiting Data Augmentation for Rotational Invariance in Convolutional Neural Networks [0.29127054707887967]
  We investigate how best to include rotational invariance in a CNN for image classification. Our experiments show that networks trained with data augmentation alone can classify rotated images nearly as well as in the normal unrotated case.
  arXiv  Detail & Related papers  (2023-10-12T15:53:24Z)
• Deep Neural Networks with Efficient Guaranteed Invariances [77.99182201815763]
  We address the problem of improving the performance and in particular the sample complexity of deep neural networks. Group-equivariant convolutions are a popular approach to obtain equivariant representations. We propose a multi-stream architecture, where each stream is invariant to a different transformation.
  arXiv  Detail & Related papers  (2023-03-02T20:44:45Z)
• The Lie Derivative for Measuring Learned Equivariance [84.29366874540217]
  We study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. We find that many violations of equivariance can be linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities. For example, transformers can be more equivariant than convolutional neural networks after training.
  arXiv  Detail & Related papers  (2022-10-06T15:20:55Z)
• Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
  Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly. A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work. It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
  arXiv  Detail & Related papers  (2022-09-26T22:22:30Z)
• Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration [77.99182201815763]
  Leveraging prior knowledge on intraclass variance due to transformations is a powerful method to improve the sample complexity of deep neural networks. We propose a novel monomial selection algorithm based on pruning methods to allow an application to more complex problems. We demonstrate the improved sample complexity on the Rotated-MNIST, SVHN and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2022-02-08T16:16:11Z)
• Implicit Equivariance in Convolutional Networks [1.911678487931003]
  Implicitly Equivariant Networks (IEN) induce equivariant in the different layers of a standard CNN model. We show IEN outperforms the state-of-the-art rotation equivariant tracking method while providing faster inference speed.
  arXiv  Detail & Related papers  (2021-11-28T14:44:17Z)
• DISCO: accurate Discrete Scale Convolutions [2.1485350418225244]
  Scale is often seen as a given, disturbing factor in many vision tasks. When doing so it is one of the factors why we need more data during learning. We aim for accurate scale-equivariant convolutional neural networks (SE-CNNs) applicable for problems where high granularity of scale and small filter sizes are required.
  arXiv  Detail & Related papers  (2021-06-04T21:48:09Z)
• Noise-Equipped Convolutional Neural Networks [15.297063646935078]
  Convolutional Neural Network (CNN) has been widely employed in image synthesis and translation tasks. When a CNN model is fed with a flat input, the transformation degrades into a scaling operation due to the spatial sharing nature of convolution kernels.
  arXiv  Detail & Related papers  (2020-12-09T09:01:45Z)
• What Does CNN Shift Invariance Look Like? A Visualization Study [87.79405274610681]
  Feature extraction with convolutional neural networks (CNNs) is a popular method to represent images for machine learning tasks. We focus on measuring and visualizing the shift invariance of extracted features from popular off-the-shelf CNN models. We conclude that features extracted from popular networks are not globally invariant, and that biases and artifacts exist within this variance.
  arXiv  Detail & Related papers  (2020-11-09T01:16:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.