SEKD: Self-Evolving Keypoint Detection and Description
- URL: http://arxiv.org/abs/2006.05077v1
- Date: Tue, 9 Jun 2020 06:56:50 GMT
- Title: SEKD: Self-Evolving Keypoint Detection and Description
- Authors: Yafei Song, Ling Cai, Jia Li, Yonghong Tian, Mingyang Li
- Abstract summary: We propose a self-supervised framework to learn an advanced local feature model from unlabeled natural images.
We benchmark the proposed method on homography estimation, relative pose estimation, and structure-from-motion tasks.
We will release our code along with the trained model publicly.
- Score: 42.114065439674036
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Researchers have attempted utilizing deep neural network (DNN) to learn novel
local features from images inspired by its recent successes on a variety of
vision tasks. However, existing DNN-based algorithms have not achieved such
remarkable progress that could be partly attributed to insufficient utilization
of the interactive characters between local feature detector and descriptor. To
alleviate these difficulties, we emphasize two desired properties, i.e.,
repeatability and reliability, to simultaneously summarize the inherent and
interactive characters of local feature detector and descriptor. Guided by
these properties, a self-supervised framework, namely self-evolving keypoint
detection and description (SEKD), is proposed to learn an advanced local
feature model from unlabeled natural images. Additionally, to have performance
guarantees, novel training strategies have also been dedicatedly designed to
minimize the gap between the learned feature and its properties. We benchmark
the proposed method on homography estimation, relative pose estimation, and
structure-from-motion tasks. Extensive experimental results demonstrate that
the proposed method outperforms popular hand-crafted and DNN-based methods by
remarkable margins. Ablation studies also verify the effectiveness of each
critical training strategy. We will release our code along with the trained
model publicly.
Related papers
- Generative Model-based Feature Knowledge Distillation for Action
Recognition [11.31068233536815]
Our paper introduces an innovative knowledge distillation framework, with the generative model for training a lightweight student model.
The efficacy of our approach is demonstrated through comprehensive experiments on diverse popular datasets.
arXiv Detail & Related papers (2023-12-14T03:55:29Z) - KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All [24.50129285997307]
We introduce a novel key-query learning strategy to enhance prompt matching efficiency and address the challenge of shifting features.
Our method empowers the model to achieve results surpassing those of current state-of-the-art approaches by a large margin of up to 20%.
arXiv Detail & Related papers (2023-11-26T20:35:19Z) - Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching [74.75284453828017]
Open-Vocabulary Keypoint Detection (OVKD) task is innovatively designed to use text prompts for identifying arbitrary keypoints across any species.
We have developed a novel framework named Open-Vocabulary Keypoint Detection with Semantic-feature Matching (KDSM)
This framework combines vision and language models, creating an interplay between language features and local keypoint visual features.
arXiv Detail & Related papers (2023-10-08T07:42:41Z) - Adaptive Local-Component-aware Graph Convolutional Network for One-shot
Skeleton-based Action Recognition [54.23513799338309]
We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition.
Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
arXiv Detail & Related papers (2022-09-21T02:33:07Z) - SHELS: Exclusive Feature Sets for Novelty Detection and Continual
Learning Without Class Boundaries [22.04165296584446]
We introduce a Sparse High-level-Exclusive, Low-level-Shared feature representation (SHELS)
SHELS encourages learning exclusive sets of high-level features and essential, shared low-level features.
We show that using SHELS for novelty detection results in statistically significant improvements over state-of-the-art OOD detection approaches.
arXiv Detail & Related papers (2022-06-28T03:09:55Z) - Locally Aggregated Feature Attribution on Natural Language Model
Understanding [12.233103741197334]
Locally Aggregated Feature Attribution (LAFA) is a novel gradient-based feature attribution method for NLP models.
Instead of relying on obscure reference tokens, it smooths gradients by aggregating similar reference texts derived from language model embeddings.
For evaluation purpose, we also design experiments on different NLP tasks including Entity Recognition and Sentiment Analysis on public datasets.
arXiv Detail & Related papers (2022-04-22T18:59:27Z) - Triggering Failures: Out-Of-Distribution detection by learning from
local adversarial attacks in Semantic Segmentation [76.2621758731288]
We tackle the detection of out-of-distribution (OOD) objects in semantic segmentation.
Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA)
We show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
arXiv Detail & Related papers (2021-08-03T17:09:56Z) - Progressive Self-Guided Loss for Salient Object Detection [102.35488902433896]
We present a progressive self-guided loss function to facilitate deep learning-based salient object detection in images.
Our framework takes advantage of adaptively aggregated multi-scale features to locate and detect salient objects effectively.
arXiv Detail & Related papers (2021-01-07T07:33:38Z) - FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data [106.76845921324704]
We propose a novel method named Feature Interaction Via Edge Search (FIVES)
FIVES formulates the task of interactive feature generation as searching for edges on the defined feature graph.
In this paper, we present our theoretical evidence that motivates us to search for useful interactive features with increasing order.
arXiv Detail & Related papers (2020-07-29T03:33:18Z) - HDD-Net: Hybrid Detector Descriptor with Mutual Interactive Learning [24.13425816781179]
Local feature extraction remains an active research area due to the advances in fields such as SLAM, 3D reconstructions, or AR applications.
We propose a method that treats both extractions independently and focuses on their interaction in the learning process.
We show improvements over the state of the art in terms of image matching on HPatches and 3D reconstruction quality while keeping on par on camera localisation tasks.
arXiv Detail & Related papers (2020-05-12T13:55:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.