RDD: Robust Feature Detector and Descriptor using Deformable Transformer

• URL: http://arxiv.org/abs/2505.08013v4
• Date: Mon, 21 Jul 2025 22:26:07 GMT
• Title: RDD: Robust Feature Detector and Descriptor using Deformable Transformer
• Authors: Gonglin Chen, Tianwen Fu, Haiwei Chen, Wenbin Teng, Hanyuan Xiao, Yajie Zhao,
• Abstract summary: We present Robust Deformable Detector (RDD), a novel and robust keypoint detector/descriptor.<n>We observed that deformable attention focuses on key locations, effectively reducing the search space complexity.<n>Our proposed method outperforms all state-of-the-art keypoint detection/description methods in sparse matching tasks.
• Score: 8.01082121187363
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As a core step in structure-from-motion and SLAM, robust feature detection and description under challenging scenarios such as significant viewpoint changes remain unresolved despite their ubiquity. While recent works have identified the importance of local features in modeling geometric transformations, these methods fail to learn the visual cues present in long-range relationships. We present Robust Deformable Detector (RDD), a novel and robust keypoint detector/descriptor leveraging the deformable transformer, which captures global context and geometric invariance through deformable self-attention mechanisms. Specifically, we observed that deformable attention focuses on key locations, effectively reducing the search space complexity and modeling the geometric invariance. Furthermore, we collected an Air-to-Ground dataset for training in addition to the standard MegaDepth dataset. Our proposed method outperforms all state-of-the-art keypoint detection/description methods in sparse matching tasks and is also capable of semi-dense matching. To ensure comprehensive evaluation, we introduce two challenging benchmarks: one emphasizing large viewpoint and scale variations, and the other being an Air-to-Ground benchmark -- an evaluation setting that has recently gaining popularity for 3D reconstruction across different altitudes.

Related papers

• Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition [63.55828203989405]
  We introduce a novel Topology-Aware Modeling (TAM) framework for Sim2Real UDA on object point clouds.<n>Our approach mitigates the domain gap by leveraging global spatial topology, characterized by low-level, high-frequency 3D structures.<n>We propose an advanced self-training strategy that combines cross-domain contrastive learning with self-training.
  arXiv  Detail & Related papers  (2025-06-26T11:53:59Z)
• Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
  We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture. We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
  arXiv  Detail & Related papers  (2024-09-20T07:41:47Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• Geometric Features Enhanced Human-Object Interaction Detection [11.513009304308724]
  We propose a novel end-to-end Transformer-style HOI detection model, i.e., geometric features enhanced HOI detector (GeoHOI) One key part of the model is a new unified self-supervised keypoint learning method named UniPointNet. GeoHOI effectively upgrades a Transformer-based HOI detector benefiting from the keypoints similarities measuring the likelihood of human-object interactions.
  arXiv  Detail & Related papers  (2024-06-26T18:52:53Z)
• S$^3$-MonoDETR: Supervised Shape&Scale-perceptive Deformable Transformer for Monocular 3D Object Detection [21.96072831561483]
  This paper proposes a novel Supervised Shape&Scale-perceptive Deformable Attention'' (S$3$-DA) module for monocular 3D object detection. Benefiting from this, S$3$-DA effectively estimates receptive fields for query points belonging to any category, enabling them to generate robust query features. Experiments on KITTI and Open datasets demonstrate that S$3$-DA significantly improves the detection accuracy.
  arXiv  Detail & Related papers  (2023-09-02T12:36:38Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints [8.390939268280235]
  Local feature extraction is a standard approach in computer vision for tackling important tasks such as image matching and retrieval. We propose DALF, a novel deformation-aware network for jointly detecting and describing keypoints. Our approach also enhances the performance of two real-world applications: deformable object retrieval and non-rigid 3D surface registration.
  arXiv  Detail & Related papers  (2023-04-02T18:01:51Z)
• Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images [15.251042369061024]
  Few-shot object detection (FSOD) relies on a large amount of labeled data for training. Scale and orientation variations of objects in remote sensing images pose significant challenges to existing FSOD methods. We propose integrating a feature pyramid network and utilizing prototype features to enhance query features.
  arXiv  Detail & Related papers  (2023-03-13T02:21:38Z)
• Hierarchical Point Attention for Indoor 3D Object Detection [111.04397308495618]
  This work proposes two novel attention operations as generic hierarchical designs for point-based transformer detectors. First, we propose Multi-Scale Attention (MS-A) that builds multi-scale tokens from a single-scale input feature to enable more fine-grained feature learning. Second, we propose Size-Adaptive Local Attention (Local-A) with adaptive attention regions for localized feature aggregation within bounding box proposals.
  arXiv  Detail & Related papers  (2023-01-06T18:52:12Z)
• Robust Change Detection Based on Neural Descriptor Fields [53.111397800478294]
  We develop an object-level online change detection approach that is robust to partially overlapping observations and noisy localization results. By associating objects via shape code similarity and comparing local object-neighbor spatial layout, our proposed approach demonstrates robustness to low observation overlap and localization noises.
  arXiv  Detail & Related papers  (2022-08-01T17:45:36Z)
• Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
  We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors. Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf) For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
  arXiv  Detail & Related papers  (2021-04-16T21:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.