SceneGlue: Scene-Aware Transformer for Feature Matching without Scene-Level Annotation

• URL: http://arxiv.org/abs/2604.13941v1
• Date: Wed, 15 Apr 2026 14:52:10 GMT
• Title: SceneGlue: Scene-Aware Transformer for Feature Matching without Scene-Level Annotation
• Abstract summary: Local feature matching plays a critical role in understanding the correspondence between cross-view images.<n>Traditional methods are constrained by the inherent local nature of feature descriptors.<n>We introduce SceneGlue, a scene-aware feature matching framework designed to overcome these limitations.
• Score: 54.91562048137692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Local feature matching plays a critical role in understanding the correspondence between cross-view images. However, traditional methods are constrained by the inherent local nature of feature descriptors, limiting their ability to capture non-local scene information that is essential for accurate cross-view correspondence. In this paper, we introduce SceneGlue, a scene-aware feature matching framework designed to overcome these limitations. SceneGlue leverages a hybridizable matching paradigm that integrates implicit parallel attention and explicit cross-view visibility estimation. The parallel attention mechanism simultaneously exchanges information among local descriptors within and across images, enhancing the scene's global context. To further enrich the scene awareness, we propose the Visibility Transformer, which explicitly categorizes features into visible and invisible regions, providing an understanding of cross-view scene visibility. By combining explicit and implicit scene-level awareness, SceneGlue effectively compensates for the local descriptor constraints. Notably, SceneGlue is trained using only local feature matches, without requiring scene-level groundtruth annotations. This scene-aware approach not only improves accuracy and robustness but also enhances interpretability compared to traditional methods. Extensive experiments on applications such as homography estimation, pose estimation, image matching, and visual localization validate SceneGlue's superior performance. The source code is available at https://github.com/songlin-du/SceneGlue.

Related papers

• SG2Loc: Sequential Visual Localization on 3D Scene Graphs [108.42363585468458]
  This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs.<n>By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world datasets.
  arXiv  Detail & Related papers  (2026-06-10T10:03:21Z)
• Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework [16.961220047066792]
  ViLaCo is a vision-language collaborative reasoning framework that introduces auxiliary semantic supervision distilled from pre-trained vision-language models.<n>ViLaCo substantially outperforms existing WSIFL methods, achieving state-of-the-art performance in both detection and localization accuracy.
  arXiv  Detail & Related papers  (2025-08-02T12:14:29Z)
• Context-Based Visual-Language Place Recognition [4.737519767218666]
  A popular approach to vision-based place recognition relies on low-level visual features. We introduce a novel VPR approach that remains robust to scene changes and does not require additional training. Our method constructs semantic image descriptors by extracting pixel-level embeddings using a zero-shot, language-driven semantic segmentation model.
  arXiv  Detail & Related papers  (2024-10-25T06:59:11Z)
• Scene Graph Generation with Role-Playing Large Language Models [50.252588437973245]
  Current approaches for open-vocabulary scene graph generation (OVSGG) use vision-language models such as CLIP. We propose SDSGG, a scene-specific description based OVSGG framework. To capture the complicated interplay between subjects and objects, we propose a new lightweight module called mutual visual adapter.
  arXiv  Detail & Related papers  (2024-10-20T11:40:31Z)
• Breaking the Frame: Visual Place Recognition by Overlap Prediction [53.17564423756082]
  We propose a novel visual place recognition approach based on overlap prediction, called VOP.<n>VOP proceeds co-visible image sections by obtaining patch-level embeddings using a Vision Transformer backbone.<n>Our approach uses a voting mechanism to assess overlap scores for potential database images.
  arXiv  Detail & Related papers  (2024-06-23T20:00:20Z)
• Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement [32.335953514942474]
  This paper proposes to jointly learn the scene representation along with a 3D dense feature field and a 2D feature extractor. We learn the underlying geometry of the scene with an implicit field through volumetric rendering and design our feature field to leverage intermediate geometric information encoded in the implicit field. Visual localization is then achieved by aligning the image-based features and the rendered volumetric features.
  arXiv  Detail & Related papers  (2024-06-12T17:51:53Z)
• Context Disentangling and Prototype Inheriting for Robust Visual Grounding [56.63007386345772]
  Visual grounding (VG) aims to locate a specific target in an image based on a given language query. We propose a novel framework with context disentangling and prototype inheriting for robust visual grounding to handle both scenes. Our method outperforms the state-of-the-art methods in both scenarios.
  arXiv  Detail & Related papers  (2023-12-19T09:03:53Z)
• Multi-level Cross-modal Feature Alignment via Contrastive Learning towards Zero-shot Classification of Remote Sensing Image Scenes [7.17717863134783]
  Cross-modal feature alignment methods have been proposed to address the zero-shot image scene classification. We propose a multi-level cross-modal feature alignment method via contrastive learning for zero-shot classification of remote sensing image scenes. Our proposed method outperforms state of the art methods for zero-shot remote sensing image scene classification.
  arXiv  Detail & Related papers  (2023-05-31T10:00:45Z)
• CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion [83.30168660888913]
  We present CommonScenes, a fully generative model that converts scene graphs into corresponding controllable 3D scenes. Our pipeline consists of two branches, one predicting the overall scene layout via a variational auto-encoder and the other generating compatible shapes. The generated scenes can be manipulated by editing the input scene graph and sampling the noise in the diffusion model.
  arXiv  Detail & Related papers  (2023-05-25T17:39:13Z)
• Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark [46.166955777187816]
  This paper focuses on understanding the role of image retrieval for multiple visual localization paradigms. We introduce a novel benchmark setup and compare state-of-the-art retrieval representations on multiple datasets. Using these tools and in-depth analysis, we show that retrieval performance on classical landmark retrieval or place recognition tasks correlates only for some but not all paradigms to localization performance.
  arXiv  Detail & Related papers  (2022-05-31T12:59:01Z)
• SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor [51.298760338410624]
  We propose a SceneEncoder module to impose a scene-aware guidance to enhance the effect of global information. The module predicts a scene descriptor, which learns to represent the categories of objects existing in the scene. We also design a region similarity loss to propagate distinguishing features to their own neighboring points with the same label.
  arXiv  Detail & Related papers  (2020-01-24T16:53:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.