Rigidity Preserving Image Transformations and Equivariance in Perspective

• URL: http://arxiv.org/abs/2201.13065v1
• Date: Mon, 31 Jan 2022 08:43:10 GMT
• Title: Rigidity Preserving Image Transformations and Equivariance in Perspective
• Authors: Lucas Brynte, Georg B\"okman, Axel Flinth, Fredrik Kahl
• Abstract summary: We characterize the class of image plane transformations which realize rigid camera motions and call these transformations rigidity preserving' In particular, 2D translations of pinhole images are not rigidity preserving.
• Score: 15.261790674845562
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We characterize the class of image plane transformations which realize rigid camera motions and call these transformations `rigidity preserving'. In particular, 2D translations of pinhole images are not rigidity preserving. Hence, when using CNNs for 3D inference tasks, it can be beneficial to modify the inductive bias from equivariance towards translations to equivariance towards rigidity preserving transformations. We investigate how equivariance with respect to rigidity preserving transformations can be approximated in CNNs, and test our ideas on both 6D object pose estimation and visual localization. Experimentally, we improve on several competitive baselines.

Related papers

• Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection [37.142470149311904]
  We propose atemporal equivariant learning framework by considering both spatial and temporal augmentations jointly. We show our pre-training method for 3D object detection which outperforms existing equivariant and invariant approaches in many settings.
  arXiv  Detail & Related papers  (2024-04-17T20:41:49Z)
• Equivariant Similarity for Vision-Language Foundation Models [134.77524524140168]
  This study focuses on the multimodal similarity function that is not only the major training objective but also the core delivery to support downstream tasks. We propose EqSim, a regularization loss that can be efficiently calculated from any two matched training pairs. Compared to the existing evaluation sets, EqBen is the first to focus on "visual-minimal change"
  arXiv  Detail & Related papers  (2023-03-25T13:22:56Z)
• Image Deblurring by Exploring In-depth Properties of Transformer [86.7039249037193]
  We leverage deep features extracted from a pretrained vision transformer (ViT) to encourage recovered images to be sharp without sacrificing the performance measured by the quantitative metrics. By comparing the transformer features between recovered image and target one, the pretrained transformer provides high-resolution blur-sensitive semantic information. One regards the features as vectors and computes the discrepancy between representations extracted from recovered image and target one in Euclidean space.
  arXiv  Detail & Related papers  (2023-03-24T14:14:25Z)
• Learning Transformations To Reduce the Geometric Shift in Object Detection [60.20931827772482]
  We tackle geometric shifts emerging from variations in the image capture process. We introduce a self-training approach that learns a set of geometric transformations to minimize these shifts. We evaluate our method on two different shifts, i.e., a camera's field of view (FoV) change and a viewpoint change.
  arXiv  Detail & Related papers  (2023-01-13T11:55:30Z)
• The Lie Derivative for Measuring Learned Equivariance [84.29366874540217]
  We study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. We find that many violations of equivariance can be linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities. For example, transformers can be more equivariant than convolutional neural networks after training.
  arXiv  Detail & Related papers  (2022-10-06T15:20:55Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• Quantised Transforming Auto-Encoders: Achieving Equivariance to Arbitrary Transformations in Deep Networks [23.673155102696338]
  Convolutional Neural Networks (CNNs) are equivariant to image translation. We propose an auto-encoder architecture whose embedding obeys an arbitrary set of equivariance relations simultaneously. We demonstrate results of successful re-rendering of transformed versions of input images on several datasets.
  arXiv  Detail & Related papers  (2021-11-25T02:26:38Z)
• Self-Supervised 3D Hand Pose Estimation from monocular RGB via Contrastive Learning [50.007445752513625]
  We propose a new self-supervised method for the structured regression task of 3D hand pose estimation. We experimentally investigate the impact of invariant and equivariant contrastive objectives. We show that a standard ResNet-152, trained on additional unlabeled data, attains an improvement of $7.6%$ in PA-EPE on FreiHAND.
  arXiv  Detail & Related papers  (2021-06-10T17:48:57Z)
• Grounding inductive biases in natural images:invariance stems from variations in data [20.432568247732206]
  We study the factors of variation in a real dataset, ImageNet. We show standard augmentation relies on a precise combination of translation and scale. We find that the main factors of variation in ImageNet mostly relate to appearance.
  arXiv  Detail & Related papers  (2021-06-09T14:58:57Z)
• Learning Equivariant Representations [10.745691354609738]
  Convolutional neural networks (CNNs) are successful examples of this principle. We propose equivariant models for different transformations defined by groups of symmetries. These models leverage symmetries in the data to reduce sample and model complexity and improve generalization performance.
  arXiv  Detail & Related papers  (2020-12-04T18:46:17Z)
• Certified Defense to Image Transformations via Randomized Smoothing [13.482057387436342]
  We extend randomized smoothing to cover transformations (e.g., rotations, translations) and certify in the parameter space. This is particularly challenging as perturbed and rounding effects mean that image transformations do not compose. We show how individual certificates can be obtained via either statistical error bounds or efficient inverse computation of the image transformation.
  arXiv  Detail & Related papers  (2020-02-27T22:02:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.