DISK: Learning local features with policy gradient

• URL: http://arxiv.org/abs/2006.13566v2
• Date: Tue, 27 Oct 2020 10:32:54 GMT
• Title: DISK: Learning local features with policy gradient
• Authors: Micha{\l} J. Tyszkiewicz, Pascal Fua, Eduard Trulls
• Abstract summary: Local feature frameworks are difficult to learn in an end-to-end fashion, due to the discreteness inherent to the selection and matching of sparse keypoints. We introduce DISK (DIScrete Keypoints), a novel method that overcomes these obstacles by leveraging principles from Reinforcement Learning (RL) Our simple yet expressive probabilistic model lets us keep the training and inference regimes close, while maintaining good enough convergence properties to reliably train from scratch.
• Score: 63.12124363163665
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Local feature frameworks are difficult to learn in an end-to-end fashion, due to the discreteness inherent to the selection and matching of sparse keypoints. We introduce DISK (DIScrete Keypoints), a novel method that overcomes these obstacles by leveraging principles from Reinforcement Learning (RL), optimizing end-to-end for a high number of correct feature matches. Our simple yet expressive probabilistic model lets us keep the training and inference regimes close, while maintaining good enough convergence properties to reliably train from scratch. Our features can be extracted very densely while remaining discriminative, challenging commonly held assumptions about what constitutes a good keypoint, as showcased in Fig. 1, and deliver state-of-the-art results on three public benchmarks.

Related papers

• SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models [80.90808879991182]
  Drawing on systematic analysis, we uncover two fundamental limitations of foundation models for sketch understanding. We address these limitations by strategically combining SD with CLIP, whose strong semantic understanding naturally compensates for SD's spatial-frequency biases. By dynamically injecting CLIP features into SD's denoising process and adaptively aggregating features across semantic levels, our method achieves state-of-the-art performance in sketch retrieval.
  arXiv  Detail & Related papers  (2025-03-18T10:47:46Z)
• Spatial regularisation for improved accuracy and interpretability in keypoint-based registration [5.286949071316761]
  Recent approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Here, we propose a three-fold loss to regularise the spatial distribution of the features. Our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks.
  arXiv  Detail & Related papers  (2025-03-06T14:48:25Z)
• ICL-TSVD: Bridging Theory and Practice in Continual Learning with Pre-trained Models [103.45785408116146]
  Continual learning (CL) aims to train a model that can solve multiple tasks presented sequentially. Recent CL approaches have achieved strong performance by leveraging large pre-trained models that generalize well to downstream tasks. However, such methods lack theoretical guarantees, making them prone to unexpected failures. We bridge this gap by integrating an empirically strong approach into a principled framework, designed to prevent forgetting.
  arXiv  Detail & Related papers  (2024-10-01T12:58:37Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• Beyond Prototypes: Semantic Anchor Regularization for Better Representation Learning [82.29761875805369]
  One of the ultimate goals of representation learning is to achieve compactness within a class and well-separability between classes. We propose a novel perspective to use pre-defined class anchors serving as feature centroid to unidirectionally guide feature learning. The proposed Semantic Anchor Regularization (SAR) can be used in a plug-and-play manner in the existing models.
  arXiv  Detail & Related papers  (2023-12-19T05:52:38Z)
• Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants [166.916517335816]
  In this paper, we offer a unified solution to the misalignment dilemma in the three tasks. We propose neural collapse terminus that is a fixed structure with the maximal equiangular inter-class separation for the whole label space. Our method holds the neural collapse optimality in an incremental fashion regardless of data imbalance or data scarcity.
  arXiv  Detail & Related papers  (2023-08-03T13:09:59Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• SRFeat: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence [36.44119664239748]
  We present a learning-based framework that combines the local accuracy of contrastive learning with the global consistency of geometric approaches. Our framework is general and is applicable to local feature learning in both the 3D and 2D domains.
  arXiv  Detail & Related papers  (2022-09-16T09:11:12Z)
• PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency [38.93610732090426]
  We present PointDSC, a novel deep neural network that explicitly incorporates spatial consistency for pruning outlier correspondences. Our method outperforms the state-of-the-art hand-crafted and learning-based outlier rejection approaches on several real-world datasets.
  arXiv  Detail & Related papers  (2021-03-09T14:56:08Z)
• Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network [16.259673823482665]
  Variational Deep Embedding achieves great success in various clustering tasks. VaDE suffers from two problems: 1) it is fragile to the input noise; 2) it ignores the locality information between the neighboring data points. We propose a joint learning framework that improves VaDE with a robust embedding discriminator and a local structure constraint.
  arXiv  Detail & Related papers  (2020-12-25T02:31:55Z)
• Better scalability under potentially heavy-tailed feedback [6.903929927172917]
  We study scalable alternatives to robust gradient descent (RGD) techniques that can be used when the losses and/or gradients can be heavy-tailed. We focus computational effort on robustly choosing a strong candidate based on a collection of cheap sub-processes which can be run in parallel. The exact selection process depends on the convexity of the underlying objective, but in all cases, our selection technique amounts to a robust form of boosting the confidence of weak learners.
  arXiv  Detail & Related papers  (2020-12-14T08:56:04Z)
• PointNetLK Revisited [37.594591809918185]
  We show that PointNetLK can exhibit remarkable generalization properties while reaping the inherent fidelity benefits of a learning framework. Our approach not only outperforms the state-of-the-art in mismatched conditions but also produces results competitive with current learning methods.
  arXiv  Detail & Related papers  (2020-08-21T15:09:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.