SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching

• URL: http://arxiv.org/abs/2606.03406v1
• Date: Tue, 02 Jun 2026 09:47:59 GMT
• Title: SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching
• Abstract summary: We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling.<n>It introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment.<n>Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning.
• Score: 25.84764820920445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher

Related papers

• Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives [74.46417373478194]
  Cross-view feature matching aims to establish reliable correspondences across images with large viewpoint variations.<n>Over the past decade, the field has evolved from task-specific models toward increasingly unified and generalizable correspondence models.<n>Despite these advances, existing studies remain highly diverse in their problem formulations, model architectures, training paradigms, and evaluation protocols.
  arXiv  Detail & Related papers  (2026-08-11T16:00:40Z)
• COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relations [73.40758654107724]
  We introduce a COMposed image retrieval network guided by By attrIbute-based NEighbor Relations (COMBINER)<n>Specifically, we first design an Adaptive Semantic Disentanglement module, which is capable of disentangling attribute features.<n> Secondly, we propose a Unified Prototype-based Composition module, which can construct cross-modal unified prototypes.<n>Finally, we introduce a Dual Relations Modeling module, which can mine pairwise and neighbor relations based on attribute similarity.
  arXiv  Detail & Related papers  (2026-06-03T08:43:51Z)
• Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning [17.753167530453997]
  We introduce Spatial Prediction, a spatially aware pretext regression task that predicts the relative position and scale between a pair of disentangled local views from the same image.<n>By modeling part-to-part relationships in a continuous geometric space, SP encourages representations to capture fine-grained spatial dependencies.<n>Experiments show consistent improvements across image recognition, fine-grained classification, semantic segmentation, and depth estimation, as well as substantial gains in out-of-distribution for object recognition.
  arXiv  Detail & Related papers  (2026-05-11T04:15:34Z)
• Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation [23.546777614096424]
  Few-shot segmentation (FSS) has gained significant attention for its ability to generalize to novel classes with limited supervision.<n>This paper tackles these challenges by introducing VINE, a unified framework that models structural consistency and discrimination to refine class-specific prototypes.
  arXiv  Detail & Related papers  (2026-03-06T06:36:42Z)
• CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval [54.15776146365823]
  Composed Image Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text.<n>We propose CSMCIR, a unified representation framework that achieves efficient query-target alignment through three synergistic components.
  arXiv  Detail & Related papers  (2026-01-07T09:21:38Z)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation [120.23172120151821]
  We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models.<n>We introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences.<n>We propose a new metric, Visual Semantic Matching, that quantifies visual inconsistencies in subject-driven image generation.
  arXiv  Detail & Related papers  (2025-09-26T07:11:55Z)
• PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models [80.65273820998875]
  We present PanMatch, a versatile foundation model for robust correspondence matching.<n>Our key insight is that any two-frame correspondence matching task can be addressed within a 2D displacement estimation framework.<n>PanMatch achieves multi-task integration by endowing displacement estimation algorithms with unprecedented generalization capabilities.
  arXiv  Detail & Related papers  (2025-07-11T08:18:52Z)
• Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models [85.51753014478315]
  We introduce AdaptPrune, a novel plug-and-play training-free pruning method.<n>It builds on conventional attention-based pruning by integrating spatial distance and token similarity with an adaptive NMS approach.<n>Our approach ensures a comprehensive evaluation of token importance and substantially refines the pruning decisions.
  arXiv  Detail & Related papers  (2025-03-11T03:58:17Z)
• Medical Image Registration Meets Vision Foundation Model: Prototype Learning and Contour Awareness [11.671950446844356]
  Existing deformable registration methods rely solely on intensity-based similarity metrics, lacking explicit anatomical knowledge.<n>We propose a novel SAM-assisted registration framework incorporating prototype learning and contour awareness.<n>Our framework significantly outperforms existing methods across multiple datasets.
  arXiv  Detail & Related papers  (2025-02-17T04:54:47Z)
• Towards Fine-grained Interactive Segmentation in Images and Videos [21.22536962888316]
  We present an SAM2Refiner framework built upon the SAM2 backbone.<n>This architecture allows SAM2 to generate fine-grained segmentation masks for both images and videos.<n>In addition, a mask refinement module is devised by employing a multi-scale cascaded structure to fuse mask features with hierarchical representations from the encoder.
  arXiv  Detail & Related papers  (2025-02-12T06:38:18Z)
• SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketches [116.1810651297801]
  SketchYourSeg establishes freehand sketches as a powerful query modality for subjective image segmentation.<n>Our evaluations demonstrate superior performance over existing approaches across diverse benchmarks.
  arXiv  Detail & Related papers  (2025-01-27T13:07:51Z)
• Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation [50.433911327489554]
  The goal of referring remote sensing image segmentation (RRSIS) is to generate a pixel-level mask of the target object identified by the referring expression.<n>To address the aforementioned challenges, a novel RRSIS framework is proposed, termed the cross-modal bidirectional interaction model (CroBIM)<n>To further forster the research of RRSIS, we also construct RISBench, a new large-scale benchmark dataset comprising 52,472 image-language-label triplets.
  arXiv  Detail & Related papers  (2024-10-11T08:28:04Z)
• GMMLoc: Structure Consistent Visual Localization with Gaussian Mixture Models [23.72910988500612]
  We present a method that tracks a camera in a prior map modelled by the Gaussian Mixture Model (GMM) With the pose estimated by the front-end initially, the local visual observations and map components are associated efficiently. We show how our system can provide a centimeter-level localization accuracy with only trivial computational overhead.
  arXiv  Detail & Related papers  (2020-06-24T12:41:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.