Just a Matter of Scale? Reevaluating Scale Equivariance in Convolutional Neural Networks

• URL: http://arxiv.org/abs/2211.10288v1
• Date: Fri, 18 Nov 2022 15:27:05 GMT
• Title: Just a Matter of Scale? Reevaluating Scale Equivariance in Convolutional Neural Networks
• Authors: Thomas Altstidl, An Nguyen, Leo Schwinn, Franz K\"oferl, Christopher Mutschler, Bj\"orn Eskofier, Dario Zanca
• Abstract summary: Convolutional networks are not equivariant to variations in scale and fail to generalize to objects of different sizes. We introduce a new family of models that applies many re-scaled kernels with shared weights in parallel and then selects the most appropriate one. Our experimental results on STIR show that both the existing and proposed approaches can improve generalization across scales compared to standard convolutions.
• Score: 3.124871781422893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The widespread success of convolutional neural networks may largely be attributed to their intrinsic property of translation equivariance. However, convolutions are not equivariant to variations in scale and fail to generalize to objects of different sizes. Despite recent advances in this field, it remains unclear how well current methods generalize to unobserved scales on real-world data and to what extent scale equivariance plays a role. To address this, we propose the novel Scaled and Translated Image Recognition (STIR) benchmark based on four different domains. Additionally, we introduce a new family of models that applies many re-scaled kernels with shared weights in parallel and then selects the most appropriate one. Our experimental results on STIR show that both the existing and proposed approaches can improve generalization across scales compared to standard convolutions. We also demonstrate that our family of models is able to generalize well towards larger scales and improve scale equivariance. Moreover, due to their unique design we can validate that kernel selection is consistent with input scale. Even so, none of the evaluated models maintain their performance for large differences in scale, demonstrating that a general understanding of how scale equivariance can improve generalization and robustness is still lacking.

Related papers

• Riesz feature representation: scale equivariant scattering network for classification tasks [0.6827423171182154]
  Scattering networks yield powerful hierarchical image descriptors which do not require lengthy training. They rely on sampling the scale dimension. In this work, we define an alternative feature representation based on the Riesz transform.
  arXiv  Detail & Related papers  (2023-07-17T13:21:28Z)
• What Affects Learned Equivariance in Deep Image Recognition Models? [10.590129221143222]
  We find evidence for a correlation between learned translation equivariance and validation accuracy on ImageNet. Data augmentation, reduced model capacity and inductive bias in the form of convolutions induce higher learned equivariance in neural networks.
  arXiv  Detail & Related papers  (2023-04-05T17:54:25Z)
• The Lie Derivative for Measuring Learned Equivariance [84.29366874540217]
  We study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. We find that many violations of equivariance can be linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities. For example, transformers can be more equivariant than convolutional neural networks after training.
  arXiv  Detail & Related papers  (2022-10-06T15:20:55Z)
• On the Strong Correlation Between Model Invariance and Generalization [54.812786542023325]
  Generalization captures a model's ability to classify unseen data. Invariance measures consistency of model predictions on transformations of the data. From a dataset-centric view, we find a certain model's accuracy and invariance linearly correlated on different test sets.
  arXiv  Detail & Related papers  (2022-07-14T17:08:25Z)
• Do Deep Networks Transfer Invariances Across Classes? [123.84237389985236]
  We show how a generative approach for learning the nuisance transformations can help transfer invariances across classes. Our results provide one explanation for why classifiers generalize poorly on unbalanced and longtailed distributions.
  arXiv  Detail & Related papers  (2022-03-18T04:38:18Z)
• Regularising for invariance to data augmentation improves supervised learning [82.85692486314949]
  We show that using multiple augmentations per input can improve generalisation. We propose an explicit regulariser that encourages this invariance on the level of individual model predictions.
  arXiv  Detail & Related papers  (2022-03-07T11:25:45Z)
• DISCO: accurate Discrete Scale Convolutions [2.1485350418225244]
  Scale is often seen as a given, disturbing factor in many vision tasks. When doing so it is one of the factors why we need more data during learning. We aim for accurate scale-equivariant convolutional neural networks (SE-CNNs) applicable for problems where high granularity of scale and small filter sizes are required.
  arXiv  Detail & Related papers  (2021-06-04T21:48:09Z)
• Learning Invariances in Neural Networks [51.20867785006147]
  We show how to parameterize a distribution over augmentations and optimize the training loss simultaneously with respect to the network parameters and augmentation parameters. We can recover the correct set and extent of invariances on image classification, regression, segmentation, and molecular property prediction from a large space of augmentations.
  arXiv  Detail & Related papers  (2020-10-22T17:18:48Z)
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data [52.78581260260455]
  We propose a general method to construct a convolutional layer that is equivariant to transformations from any specified Lie group. We apply the same model architecture to images, ball-and-stick molecular data, and Hamiltonian dynamical systems.
  arXiv  Detail & Related papers  (2020-02-25T17:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.