FeatureBooster: Boosting Feature Descriptors with a Lightweight Neural Network

• URL: http://arxiv.org/abs/2211.15069v3
• Date: Tue, 28 Mar 2023 06:34:08 GMT
• Title: FeatureBooster: Boosting Feature Descriptors with a Lightweight Neural Network
• Authors: Xinjiang Wang, Zeyu Liu, Yu Hu, Wei Xi, Wenxian Yu, Danping Zou
• Abstract summary: We introduce a lightweight network to improve descriptors of keypoints within the same image. The network takes the original descriptors and the geometric properties of keypoints as the input. We use the proposed network to boost both hand-crafted (ORB, SIFT) and the state-of-the-art learning-based descriptors.
• Score: 16.10404845106396
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a lightweight network to improve descriptors of keypoints within the same image. The network takes the original descriptors and the geometric properties of keypoints as the input, and uses an MLP-based self-boosting stage and a Transformer-based cross-boosting stage to enhance the descriptors. The boosted descriptors can be either real-valued or binary ones. We use the proposed network to boost both hand-crafted (ORB, SIFT) and the state-of-the-art learning-based descriptors (SuperPoint, ALIKE) and evaluate them on image matching, visual localization, and structure-from-motion tasks. The results show that our method significantly improves the performance of each task, particularly in challenging cases such as large illumination changes or repetitive patterns. Our method requires only 3.2ms on desktop GPU and 27ms on embedded GPU to process 2000 features, which is fast enough to be applied to a practical system. The code and trained weights are publicly available at github.com/SJTU-ViSYS/FeatureBooster.

Related papers

• Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence [12.602194710071116]
  This paper presents a new method that uses semantic cues from foundation vision model features to enhance local feature matching. We present adapted versions of six existing descriptors, with an average increase in performance of 29% in camera localization.
  arXiv  Detail & Related papers  (2024-10-12T13:45:26Z)
• Residual Learning for Image Point Descriptors [56.917951170421894]
  We propose a very simple and effective approach to learning local image descriptors by using a hand-crafted detector and descriptor. We optimize the final descriptor by leveraging the knowledge already present in the handcrafted descriptor. Our approach has potential applications in ensemble learning and learning with non-differentiable functions.
  arXiv  Detail & Related papers  (2023-12-24T12:51:30Z)
• Dynamic Perceiver for Efficient Visual Recognition [87.08210214417309]
  We propose Dynamic Perceiver (Dyn-Perceiver) to decouple the feature extraction procedure and the early classification task. A feature branch serves to extract image features, while a classification branch processes a latent code assigned for classification tasks. Early exits are placed exclusively within the classification branch, thus eliminating the need for linear separability in low-level features.
  arXiv  Detail & Related papers  (2023-06-20T03:00:22Z)
• ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation [27.04762347838776]
  We propose the Sparse Deformable Descriptor Head (SDDH), which learns the deformable positions of supporting features for each keypoint and constructs deformable descriptors. We show that the proposed network is both efficient and powerful in various visual measurement tasks, including image matching, 3D reconstruction, and visual relocalization.
  arXiv  Detail & Related papers  (2023-04-07T12:05:39Z)
• Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors [101.62384271200169]
  A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks. We investigate multi-layer perceptrons (MLPs) to extract low-dimensional but high-quality descriptors. We consider different applications, including visual localization, patch verification, image matching and retrieval.
  arXiv  Detail & Related papers  (2022-09-27T17:59:04Z)
• ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretization [71.91942002659795]
  We investigate and adapt network quantization techniques to accelerate inference and enable its use on compute limited platforms. ZippyPoint, our efficient quantized network with binary descriptors, improves the network runtime speed, the descriptor matching speed, and the 3D model size. These improvements come at a minor performance degradation as evaluated on the tasks of homography estimation, visual localization, and map-free visual relocalization.
  arXiv  Detail & Related papers  (2022-03-07T18:59:03Z)
• Multi-level Second-order Few-shot Learning [111.0648869396828]
  We propose a Multi-level Second-order (MlSo) few-shot learning network for supervised or unsupervised few-shot image classification and few-shot action recognition. We leverage so-called power-normalized second-order base learner streams combined with features that express multiple levels of visual abstraction. We demonstrate respectable results on standard datasets such as Omniglot, mini-ImageNet, tiered-ImageNet, Open MIC, fine-grained datasets such as CUB Birds, Stanford Dogs and Cars, and action recognition datasets such as HMDB51, UCF101, and mini-MIT.
  arXiv  Detail & Related papers  (2022-01-15T19:49:00Z)
• GhostSR: Learning Ghost Features for Efficient Image Super-Resolution [49.393251361038025]
  Single image super-resolution (SISR) system based on convolutional neural networks (CNNs) achieves fancy performance while requires huge computational costs. We propose to use shift operation to generate the redundant features (i.e., Ghost features) of SISR models. We show that both the non-compact and lightweight SISR models embedded in our proposed module can achieve comparable performance to that of their baselines.
  arXiv  Detail & Related papers  (2021-01-21T10:09:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.