Extracting Deformation-Aware Local Features by Learning to Deform

• URL: http://arxiv.org/abs/2111.10617v1
• Date: Sat, 20 Nov 2021 15:46:33 GMT
• Title: Extracting Deformation-Aware Local Features by Learning to Deform
• Authors: Guilherme Potje, Renato Martins, Felipe Cadar and Erickson R. Nascimento
• Abstract summary: We present a new approach to compute features from still images that are robust to non-rigid deformations. We train the model architecture end-to-end by applying non-rigid deformations to objects in a simulated environment. Experiments show that our method outperforms state-of-the-art handcrafted, learning-based image, and RGB-D descriptors in different datasets.
• Score: 3.364554138758565
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Despite the advances in extracting local features achieved by handcrafted and learning-based descriptors, they are still limited by the lack of invariance to non-rigid transformations. In this paper, we present a new approach to compute features from still images that are robust to non-rigid deformations to circumvent the problem of matching deformable surfaces and objects. Our deformation-aware local descriptor, named DEAL, leverages a polar sampling and a spatial transformer warping to provide invariance to rotation, scale, and image deformations. We train the model architecture end-to-end by applying isometric non-rigid deformations to objects in a simulated environment as guidance to provide highly discriminative local features. The experiments show that our method outperforms state-of-the-art handcrafted, learning-based image, and RGB-D descriptors in different datasets with both real and realistic synthetic deformable objects in still images. The source code and trained model of the descriptor are publicly available at https://www.verlab.dcc.ufmg.br/descriptors/neurips2021.

Related papers

• Learning Dual-Level Deformable Implicit Representation for Real-World Scale Arbitrary Super-Resolution [81.74583887661794]
  We build a new real-world super-resolution benchmark with both integer and non-integer scaling factors for the training and evaluation of real-world scale arbitrary super-resolution. Specifically, we design the appearance embedding and deformation field to handle both image-level and pixel-level deformations caused by real-world degradations. Our trained model achieves state-of-the-art performance on the RealArbiSR and RealSR benchmarks for real-world scale arbitrary super-resolution.
  arXiv  Detail & Related papers  (2024-03-16T13:44:42Z)
• KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation [87.23575166061413]
  KP-RED is a unified KeyPoint-driven REtrieval and Deformation framework. It takes object scans as input and jointly retrieves and deforms the most geometrically similar CAD models.
  arXiv  Detail & Related papers  (2024-03-15T08:44:56Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [63.54342601757723]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Learning Transformations To Reduce the Geometric Shift in Object Detection [60.20931827772482]
  We tackle geometric shifts emerging from variations in the image capture process. We introduce a self-training approach that learns a set of geometric transformations to minimize these shifts. We evaluate our method on two different shifts, i.e., a camera's field of view (FoV) change and a viewpoint change.
  arXiv  Detail & Related papers  (2023-01-13T11:55:30Z)
• RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning [62.86400614141706]
  We propose a new learning model, i.e., Rectangling Rectification Network (RecRecNet) Our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2023-01-04T15:12:57Z)
• Geo-SIC: Learning Deformable Geometric Shapes in Deep Image Classifiers [8.781861951759948]
  This paper presents Geo-SIC, the first deep learning model to learn deformable shapes in a deformation space for an improved performance of image classification. We introduce a newly designed framework that (i) simultaneously derives features from both image and latent shape spaces with large intra-class variations. We develop a boosted classification network, equipped with an unsupervised learning of geometric shape representations.
  arXiv  Detail & Related papers  (2022-10-25T01:55:17Z)
• Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian [58.704089101826774]
  We present a 3D-aware image deformation method with minimal restrictions on shape category and deformation type. We take a supervised learning-based approach to predict the shape Laplacian of the underlying volume of a 3D reconstruction represented as a point cloud. In the experiments, we present our results of deforming 2D character and clothed human images.
  arXiv  Detail & Related papers  (2022-03-29T04:57:18Z)
• Learning Geodesic-Aware Local Features from RGB-D Images [8.115075181267109]
  We propose a new approach to compute descriptors from RGB-D images that are invariant to non-rigid deformations. Our proposed description strategies are grounded on the key idea of learning feature representations on undistorted local image patches. In different experiments using real and publicly available RGB-D data benchmarks, they consistently outperforms state-of-the-art handcrafted and learning-based image and RGB-D descriptors.
  arXiv  Detail & Related papers  (2022-03-22T19:52:49Z)
• Image-to-image Transformation with Auxiliary Condition [0.0]
  We propose to introduce the label information of subjects, e.g., pose and type of objects in the training of CycleGAN, and lead it to obtain label-wise transforamtion models. We evaluate our proposed method called Label-CycleGAN, through experiments on the digit image transformation from SVHN to MNIST and the surveillance camera image transformation from simulated to real images.
  arXiv  Detail & Related papers  (2021-06-25T15:33:11Z)
• Robust Training Using Natural Transformation [19.455666609149567]
  We present NaTra, an adversarial training scheme to improve robustness of image classification algorithms. We target attributes of the input images that are independent of the class identification, and manipulate those attributes to mimic real-world natural transformations. We demonstrate the efficacy of our scheme by utilizing the disentangled latent representations derived from well-trained GANs.
  arXiv  Detail & Related papers  (2021-05-10T01:56:03Z)
• Transformation Based Deep Anomaly Detection in Astronomical Images [0.0]
  We introduce new filter based transformations useful for detecting anomalies in astronomical images. We also propose a transformation selection strategy that allows us to find indistinguishable pairs of transformations. The models were tested on astronomical images from the High Cadence Transient Survey (HiTS) and Zwicky Transient Facility (ZTF) datasets.
  arXiv  Detail & Related papers  (2020-05-15T21:02:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.