Improving the matching of deformable objects by learning to detect keypoints

• URL: http://arxiv.org/abs/2309.00434v2
• Date: Tue, 12 Sep 2023 07:18:26 GMT
• Title: Improving the matching of deformable objects by learning to detect keypoints
• Authors: Felipe Cadar and Welerson Melo and Vaishnavi Kanagasabapathi and Guilherme Potje and Renato Martins and Erickson R. Nascimento
• Abstract summary: We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. We train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method. We also apply our method on the complex real-world task object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task.
• Score: 6.4587163310833855
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. For that, we apply geometric and photometric warpings to images to generate a supervisory signal, allowing the optimization of the detector. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023

Related papers

• Learning to Make Keypoints Sub-Pixel Accurate [80.55676599677824]
  This work addresses the challenge of sub-pixel accuracy in detecting 2D local features. We propose a novel network that enhances any detector with sub-pixel precision by learning an offset vector for detected features.
  arXiv  Detail & Related papers  (2024-07-16T12:39:56Z)
• Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Images [76.47980643420375]
  This paper builds on the hypothesis that there is an inherent data-hungry matter in learning semantic correspondences. We demonstrate a simple machine annotator reliably enriches paired key points via machine supervision. Our models surpass current state-of-the-art models on semantic correspondence learning benchmarks like SPair-71k, PF-PASCAL, and PF-WILLOW.
  arXiv  Detail & Related papers  (2023-11-30T13:22:15Z)
• PrObeD: Proactive Object Detection Wrapper [15.231600709902127]
  PrObeD consists of an encoder-decoder architecture, where the encoder network generates an image-dependent signal templates to encrypt the input images. We propose a wrapper based on proactive schemes, PrObeD, which enhances the performance of these object detectors by learning a signal. Our experiments on MS-COCO, CAMO, COD$10$K, and NC$4$K datasets show improvement over different detectors after applying PrObeD.
  arXiv  Detail & Related papers  (2023-10-28T19:25:01Z)
• Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network [52.29330138835208]
  Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images. We propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide message passing.
  arXiv  Detail & Related papers  (2023-07-04T02:50:44Z)
• Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints [8.390939268280235]
  Local feature extraction is a standard approach in computer vision for tackling important tasks such as image matching and retrieval. We propose DALF, a novel deformation-aware network for jointly detecting and describing keypoints. Our approach also enhances the performance of two real-world applications: deformable object retrieval and non-rigid 3D surface registration.
  arXiv  Detail & Related papers  (2023-04-02T18:01:51Z)
• Learning to Detect Good Keypoints to Match Non-Rigid Objects in RGB Images [7.428474910083337]
  We present a novel learned keypoint detection method designed to maximize the number of correct matches for the task of non-rigid image correspondence. Our training framework uses true correspondences, obtained by matching annotated image pairs with a predefined descriptor extractor, as a ground-truth to train a convolutional neural network (CNN) Experiments show that our method outperforms the state-of-the-art keypoint detector on real images of non-rigid objects by 20 p.p. on Mean Matching Accuracy.
  arXiv  Detail & Related papers  (2022-12-13T11:59:09Z)
• LEAD: Self-Supervised Landmark Estimation by Aligning Distributions of Feature Similarity [49.84167231111667]
  Existing works in self-supervised landmark detection are based on learning dense (pixel-level) feature representations from an image. We introduce an approach to enhance the learning of dense equivariant representations in a self-supervised fashion. We show that having such a prior in the feature extractor helps in landmark detection, even under drastically limited number of annotations.
  arXiv  Detail & Related papers  (2022-04-06T17:48:18Z)
• ObjectFormer for Image Manipulation Detection and Localization [118.89882740099137]
  We propose ObjectFormer to detect and localize image manipulations. We extract high-frequency features of the images and combine them with RGB features as multimodal patch embeddings. We conduct extensive experiments on various datasets and the results verify the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-03-28T12:27:34Z)
• Data Augmentation for Object Detection via Differentiable Neural Rendering [71.00447761415388]
  It is challenging to train a robust object detector when annotated data is scarce. Existing approaches to tackle this problem include semi-supervised learning that interpolates labeled data from unlabeled data. We introduce an offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
  arXiv  Detail & Related papers  (2021-03-04T06:31:06Z)
• Robust Place Recognition using an Imaging Lidar [45.37172889338924]
  We propose a methodology for robust, real-time place recognition using an imaging lidar. Our method is truly-invariant and can tackle reverse revisiting and upside-down revisiting.
  arXiv  Detail & Related papers  (2021-03-03T01:08:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.