Learning Rotation-Equivariant Features for Visual Correspondence

• URL: http://arxiv.org/abs/2303.15472v1
• Date: Sat, 25 Mar 2023 13:42:07 GMT
• Title: Learning Rotation-Equivariant Features for Visual Correspondence
• Authors: Jongmin Lee, Byungjin Kim, Seungwook Kim, Minsu Cho
• Abstract summary: We introduce a self-supervised learning framework to extract discriminative rotation-invariant descriptors. Thanks to employing group-equivariant CNNs, our method effectively learns to obtain rotation-equivariant features and their orientations explicitly. Our method demonstrates state-of-the-art matching accuracy among existing rotation-invariant descriptors under varying rotation.
• Score: 41.79256655501003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extracting discriminative local features that are invariant to imaging variations is an integral part of establishing correspondences between images. In this work, we introduce a self-supervised learning framework to extract discriminative rotation-invariant descriptors using group-equivariant CNNs. Thanks to employing group-equivariant CNNs, our method effectively learns to obtain rotation-equivariant features and their orientations explicitly, without having to perform sophisticated data augmentations. The resultant features and their orientations are further processed by group aligning, a novel invariant mapping technique that shifts the group-equivariant features by their orientations along the group dimension. Our group aligning technique achieves rotation-invariance without any collapse of the group dimension and thus eschews loss of discriminability. The proposed method is trained end-to-end in a self-supervised manner, where we use an orientation alignment loss for the orientation estimation and a contrastive descriptor loss for robust local descriptors to geometric/photometric variations. Our method demonstrates state-of-the-art matching accuracy among existing rotation-invariant descriptors under varying rotation and also shows competitive results when transferred to the task of keypoint matching and camera pose estimation.

Related papers

• Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning [5.802794302956837]
  Euclidean deep learning is often inadequate for addressing real-world signals where the representation space is irregular and curved with complex topologies. We focus on the importance of symmetry group equivariant deep learning models and their realization of convolution-like operations on graphs, 3D shapes, and non-Euclidean spaces.
  arXiv  Detail & Related papers  (2024-09-11T15:07:18Z)
• Few-Shot Rotation-Invariant Aerial Image Semantic Segmentation [53.47160182465171]
  Few-shot aerial image segmentation is a challenging task that involves precisely parsing objects in query aerial images. The authors propose a novel few-shot rotation-invariant aerial semantic segmentation network (FRINet) Experiments demonstrate that FRINet achieves state-of-the-art performance in few-shot aerial semantic segmentation benchmark.
  arXiv  Detail & Related papers  (2023-05-29T09:28:34Z)
• 4D Panoptic Segmentation as Invariant and Equivariant Field Prediction [48.57732508537554]
  We develop rotation-equivariant neural networks for 4D panoptic segmentation. We show that our models achieve higher accuracy with lower computational costs compared to their non-equivariant counterparts. Our method sets the new state-of-the-art performance and achieves 1st place on the Semantic KITTITI 4D Panoptic leaderboard.
  arXiv  Detail & Related papers  (2023-03-28T00:20:37Z)
• Self-Supervised Learning for Group Equivariant Neural Networks [75.62232699377877]
  Group equivariant neural networks are the models whose structure is restricted to commute with the transformations on the input. We propose two concepts for self-supervised tasks: equivariant pretext labels and invariant contrastive loss. Experiments on standard image recognition benchmarks demonstrate that the equivariant neural networks exploit the proposed self-supervised tasks.
  arXiv  Detail & Related papers  (2023-03-08T08:11:26Z)
• Unsupervised Learning of Group Invariant and Equivariant Representations [10.252723257176566]
  We extend group invariant and equivariant representation learning to the field of unsupervised deep learning. We propose a general learning strategy based on an encoder-decoder framework in which the latent representation is separated in an invariant term and an equivariant group action component. The key idea is that the network learns to encode and decode data to and from a group-invariant representation by additionally learning to predict the appropriate group action to align input and output pose to solve the reconstruction task.
  arXiv  Detail & Related papers  (2022-02-15T16:44:21Z)
• Rotation-Invariant Point Convolution With Multiple Equivariant Alignments [1.0152838128195467]
  We show that using rotation-equivariant alignments, it is possible to make any convolutional layer rotation-invariant. With this core layer, we design rotation-invariant architectures which improve state-of-the-art results in both object classification and semantic segmentation.
  arXiv  Detail & Related papers  (2020-12-07T20:47:46Z)
• Invariant Deep Compressible Covariance Pooling for Aerial Scene Categorization [80.55951673479237]
  We propose a novel invariant deep compressible covariance pooling (IDCCP) to solve nuisance variations in aerial scene categorization. We conduct extensive experiments on the publicly released aerial scene image data sets and demonstrate the superiority of this method compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-11T11:13:07Z)
• A Rotation-Invariant Framework for Deep Point Cloud Analysis [132.91915346157018]
  We introduce a new low-level purely rotation-invariant representation to replace common 3D Cartesian coordinates as the network inputs. Also, we present a network architecture to embed these representations into features, encoding local relations between points and their neighbors, and the global shape structure. We evaluate our method on multiple point cloud analysis tasks, including shape classification, part segmentation, and shape retrieval.
  arXiv  Detail & Related papers  (2020-03-16T14:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.