PATS: Patch Area Transportation with Subdivision for Local Feature Matching

• URL: http://arxiv.org/abs/2303.07700v3
• Date: Tue, 23 Jan 2024 18:37:41 GMT
• Title: PATS: Patch Area Transportation with Subdivision for Local Feature Matching
• Authors: Junjie Ni, Yijin Li, Zhaoyang Huang, Hongsheng Li, Hujun Bao, Zhaopeng Cui, Guofeng Zhang
• Abstract summary: Local feature matching aims at establishing sparse correspondences between a pair of images. We propose Patch Area Transportation with Subdivision (PATS) to tackle this issue. PATS improves both matching accuracy and coverage, and shows superior performance in downstream tasks.
• Score: 78.67559513308787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Local feature matching aims at establishing sparse correspondences between a pair of images. Recently, detector-free methods present generally better performance but are not satisfactory in image pairs with large scale differences. In this paper, we propose Patch Area Transportation with Subdivision (PATS) to tackle this issue. Instead of building an expensive image pyramid, we start by splitting the original image pair into equal-sized patches and gradually resizing and subdividing them into smaller patches with the same scale. However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs. Moreover, it is hard to obtain the ground truth for real scenes. To this end, we propose patch area transportation, which enables learning scale differences in a self-supervised manner. In contrast to bipartite graph matching, which only handles one-to-one matching, our patch area transportation can deal with many-to-many relationships. PATS improves both matching accuracy and coverage, and shows superior performance in downstream tasks, such as relative pose estimation, visual localization, and optical flow estimation. The source code is available at \url{https://zju3dv.github.io/pats/}.

Related papers

• Breaking the Frame: Visual Place Recognition by Overlap Prediction [53.17564423756082]
  We propose a novel visual place recognition approach based on overlap prediction, called VOP. VOP proceeds co-visible image sections by obtaining patch-level embeddings using a Vision Transformer backbone. Our approach uses a voting mechanism to assess overlap scores for potential database images.
  arXiv  Detail & Related papers  (2024-06-23T20:00:20Z)
• Learning to Rank Patches for Unbiased Image Redundancy Reduction [80.93989115541966]
  Images suffer from heavy spatial redundancy because pixels in neighboring regions are spatially correlated. Existing approaches strive to overcome this limitation by reducing less meaningful image regions. We propose a self-supervised framework for image redundancy reduction called Learning to Rank Patches.
  arXiv  Detail & Related papers  (2024-03-31T13:12:41Z)
• 2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds [38.425876064671435]
  We propose 2D3D-MATR, a detection-free method for accurate and robust registration between images and point clouds. Our method adopts a coarse-to-fine pipeline where it first computes coarse correspondences between downsampled patches of the input image and the point cloud. To resolve the scale ambiguity in patch matching, we construct a multi-scale pyramid for each image patch and learn to find for each point patch the best matching image patch at a proper resolution level.
  arXiv  Detail & Related papers  (2023-08-10T16:10:54Z)
• Image Matching by Bare Homography [9.431261418370147]
  This paper presents Slime, a novel non-deep image matching framework which models the scene as rough local overlapping planes. Planes are mutually extended by compatible matches and the images are split into fixed tiles, with only the best homographies retained for each pair of tiles. The paper gives a thorough comparative analysis of recent state-of-the-art in image matching represented by end-to-end deep networks and hybrid pipelines.
  arXiv  Detail & Related papers  (2023-05-15T18:35:47Z)
• Generating natural images with direct Patch Distributions Matching [7.99536002595393]
  We develop an algorithm that explicitly and efficiently minimizes the distance between patch distributions in two images. Our results are often superior to single-image-GANs, require no training, and can generate high quality images in a few seconds.
  arXiv  Detail & Related papers  (2022-03-22T16:38:52Z)
• Patch-Based Stochastic Attention for Image Editing [4.8201607588546]
  We propose an efficient attention layer based on the algorithm PatchMatch, which is used for determining approximate nearest neighbors. We demonstrate the usefulness of PSAL on several image editing tasks, such as image inpainting, guided image colorization, and single-image super-resolution.
  arXiv  Detail & Related papers  (2022-02-07T13:42:00Z)
• Feature Space Targeted Attacks by Statistic Alignment [74.40447383387574]
  Feature space targeted attacks perturb images by modulating their intermediate feature maps. The current choice of pixel-wise Euclidean Distance to measure the discrepancy is questionable because it unreasonably imposes a spatial-consistency constraint on the source and target features. We propose two novel approaches called Pair-wise Alignment Attack and Global-wise Alignment Attack, which attempt to measure similarities between feature maps by high-order statistics.
  arXiv  Detail & Related papers  (2021-05-25T03:46:39Z)
• The Spatially-Correlative Loss for Various Image Translation Tasks [69.62228639870114]
  We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency. Previous methods attempt this by using pixel-level cycle-consistency or feature-level matching losses. We show distinct improvement over baseline models in all three modes of unpaired I2I translation: single-modal, multi-modal, and even single-image translation.
  arXiv  Detail & Related papers  (2021-04-02T02:13:30Z)
• SimPatch: A Nearest Neighbor Similarity Match between Image Patches [0.0]
  We try to use large patches instead of relatively small patches so that each patch contains more information. We use different feature extraction mechanisms to extract the features of each individual image patches which forms a feature matrix. The nearest patches are calculated using two different nearest neighbor algorithms in this paper for a query patch for a given image.
  arXiv  Detail & Related papers  (2020-08-07T10:51:10Z)
• Contrastive Learning for Unpaired Image-to-Image Translation [64.47477071705866]
  In image-to-image translation, each patch in the output should reflect the content of the corresponding patch in the input, independent of domain. We propose a framework based on contrastive learning to maximize mutual information between the two. We demonstrate that our framework enables one-sided translation in the unpaired image-to-image translation setting, while improving quality and reducing training time.
  arXiv  Detail & Related papers  (2020-07-30T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.