DISK: Learning local features with policy gradient

• URL: http://arxiv.org/abs/2006.13566v2
• Date: Tue, 27 Oct 2020 10:32:54 GMT
• Title: DISK: Learning local features with policy gradient
• Abstract summary: Local feature frameworks are difficult to learn in an end-to-end fashion, due to the discreteness inherent to the selection and matching of sparse keypoints. We introduce DISK (DIScrete Keypoints), a novel method that overcomes these obstacles by leveraging principles from Reinforcement Learning (RL) Our simple yet expressive probabilistic model lets us keep the training and inference regimes close, while maintaining good enough convergence properties to reliably train from scratch.
• Score: 63.12124363163665
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Local feature frameworks are difficult to learn in an end-to-end fashion, due to the discreteness inherent to the selection and matching of sparse keypoints. We introduce DISK (DIScrete Keypoints), a novel method that overcomes these obstacles by leveraging principles from Reinforcement Learning (RL), optimizing end-to-end for a high number of correct feature matches. Our simple yet expressive probabilistic model lets us keep the training and inference regimes close, while maintaining good enough convergence properties to reliably train from scratch. Our features can be extracted very densely while remaining discriminative, challenging commonly held assumptions about what constitutes a good keypoint, as showcased in Fig. 1, and deliver state-of-the-art results on three public benchmarks.

Related papers

• Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning [55.99621402404222]
  Reinforcement learning with verifiable rewards without human-annotated data, often referred to as zero RL, has emerged as a powerful paradigm for eliciting chain-of-thought reasoning.<n>We aim to elicit high-quality reasoning behaviors from the model.<n>We find that naive scaling often suffers from poor readability, token redundancy, and a lack of adaptive reasoning depth.
  arXiv  Detail & Related papers  (2026-07-14T06:14:55Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learning [12.771216755928117]
  We propose PointRFT, the first reinforcement fine-tuning paradigm tailored specifically for point cloud representation learning.<n>We show that PointRFT consistently outperforms vanilla supervised fine-tuning (SFT) across diverse benchmarks.<n>When integrated into a hybrid Pretraining-SFT-RFT paradigm, the representational capacity of point cloud foundation models is substantially unleashed.
  arXiv  Detail & Related papers  (2026-03-25T05:36:32Z)
• Regularized Latent Dynamics Prediction is a Strong Baseline For Behavioral Foundation Models [35.088440282359024]
  Behavioral Foundation Models (BFMs) produce agents with the capability to adapt to any unknown reward or task.<n>These methods are only able to produce near-optimal policies for the reward functions that are in the span of some pre-existing state features.<n>We propose an approach, Regularized Latent Dynamics Prediction (RLDP), that can match or surpass state-of-the-art complex representation learning methods for zero-shot RL.
  arXiv  Detail & Related papers  (2026-03-16T19:39:27Z)
• Shortcut Invariance: Targeted Jacobian Regularization in Disentangled Latent Space [7.8904984750896885]
  Deep neural networks are prone to learning shortcuts, spurious and easily learned correlations.<n>We present a simple and effective training method that renders the classifier functionally invariant to shortcut signals.<n>We analyze this as targeted Jacobian regularization, which forces the classifier to ignore spurious features and rely on more complex, core semantic signals.
  arXiv  Detail & Related papers  (2025-11-24T07:09:08Z)
• SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models [80.90808879991182]
  Drawing on systematic analysis, we uncover two fundamental limitations of foundation models for sketch understanding. We address these limitations by strategically combining SD with CLIP, whose strong semantic understanding naturally compensates for SD's spatial-frequency biases. By dynamically injecting CLIP features into SD's denoising process and adaptively aggregating features across semantic levels, our method achieves state-of-the-art performance in sketch retrieval.
  arXiv  Detail & Related papers  (2025-03-18T10:47:46Z)
• Spatial regularisation for improved accuracy and interpretability in keypoint-based registration [5.286949071316761]
  Recent approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Here, we propose a three-fold loss to regularise the spatial distribution of the features. Our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks.
  arXiv  Detail & Related papers  (2025-03-06T14:48:25Z)
• ICL-TSVD: Bridging Theory and Practice in Continual Learning with Pre-trained Models [103.45785408116146]
  Continual learning (CL) aims to train a model that can solve multiple tasks presented sequentially. Recent CL approaches have achieved strong performance by leveraging large pre-trained models that generalize well to downstream tasks. However, such methods lack theoretical guarantees, making them prone to unexpected failures. We bridge this gap by integrating an empirically strong approach into a principled framework, designed to prevent forgetting.
  arXiv  Detail & Related papers  (2024-10-01T12:58:37Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• Beyond Prototypes: Semantic Anchor Regularization for Better Representation Learning [82.29761875805369]
  One of the ultimate goals of representation learning is to achieve compactness within a class and well-separability between classes. We propose a novel perspective to use pre-defined class anchors serving as feature centroid to unidirectionally guide feature learning. The proposed Semantic Anchor Regularization (SAR) can be used in a plug-and-play manner in the existing models.
  arXiv  Detail & Related papers  (2023-12-19T05:52:38Z)
• Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants [166.916517335816]
  In this paper, we offer a unified solution to the misalignment dilemma in the three tasks. We propose neural collapse terminus that is a fixed structure with the maximal equiangular inter-class separation for the whole label space. Our method holds the neural collapse optimality in an incremental fashion regardless of data imbalance or data scarcity.
  arXiv  Detail & Related papers  (2023-08-03T13:09:59Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• SRFeat: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence [36.44119664239748]
  We present a learning-based framework that combines the local accuracy of contrastive learning with the global consistency of geometric approaches. Our framework is general and is applicable to local feature learning in both the 3D and 2D domains.
  arXiv  Detail & Related papers  (2022-09-16T09:11:12Z)
• PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency [38.93610732090426]
  We present PointDSC, a novel deep neural network that explicitly incorporates spatial consistency for pruning outlier correspondences. Our method outperforms the state-of-the-art hand-crafted and learning-based outlier rejection approaches on several real-world datasets.
  arXiv  Detail & Related papers  (2021-03-09T14:56:08Z)
• Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network [16.259673823482665]
  Variational Deep Embedding achieves great success in various clustering tasks. VaDE suffers from two problems: 1) it is fragile to the input noise; 2) it ignores the locality information between the neighboring data points. We propose a joint learning framework that improves VaDE with a robust embedding discriminator and a local structure constraint.
  arXiv  Detail & Related papers  (2020-12-25T02:31:55Z)
• Better scalability under potentially heavy-tailed feedback [6.903929927172917]
  We study scalable alternatives to robust gradient descent (RGD) techniques that can be used when the losses and/or gradients can be heavy-tailed. We focus computational effort on robustly choosing a strong candidate based on a collection of cheap sub-processes which can be run in parallel. The exact selection process depends on the convexity of the underlying objective, but in all cases, our selection technique amounts to a robust form of boosting the confidence of weak learners.
  arXiv  Detail & Related papers  (2020-12-14T08:56:04Z)
• PointNetLK Revisited [37.594591809918185]
  We show that PointNetLK can exhibit remarkable generalization properties while reaping the inherent fidelity benefits of a learning framework. Our approach not only outperforms the state-of-the-art in mismatched conditions but also produces results competitive with current learning methods.
  arXiv  Detail & Related papers  (2020-08-21T15:09:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.