Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence

• URL: http://arxiv.org/abs/2210.02689v1
• Date: Thu, 6 Oct 2022 05:38:27 GMT
• Title: Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence
• Authors: Sunghwan Hong, Jisu Nam, Seokju Cho, Susung Hong, Sangryul Jeon, Dongbo Min, Seungryong Kim
• Abstract summary: We present a novel method for semantic correspondence, called Neural Matching Field (NeMF) We learn a high-dimensional matching field, since a naive exhaustive inference would require querying from all pixels in the 4D space to infer pixel-wise correspondences. With these combined, competitive results are attained on several standard benchmarks for semantic correspondence.
• Score: 41.39740414165091
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing pipelines of semantic correspondence commonly include extracting high-level semantic features for the invariance against intra-class variations and background clutters. This architecture, however, inevitably results in a low-resolution matching field that additionally requires an ad-hoc interpolation process as a post-processing for converting it into a high-resolution one, certainly limiting the overall performance of matching results. To overcome this, inspired by recent success of implicit neural representation, we present a novel method for semantic correspondence, called Neural Matching Field (NeMF). However, complicacy and high-dimensionality of a 4D matching field are the major hindrances, which we propose a cost embedding network to process a coarse cost volume to use as a guidance for establishing high-precision matching field through the following fully-connected network. Nevertheless, learning a high-dimensional matching field remains challenging mainly due to computational complexity, since a naive exhaustive inference would require querying from all pixels in the 4D space to infer pixel-wise correspondences. To overcome this, we propose adequate training and inference procedures, which in the training phase, we randomly sample matching candidates and in the inference phase, we iteratively performs PatchMatch-based inference and coordinate optimization at test time. With these combined, competitive results are attained on several standard benchmarks for semantic correspondence. Code and pre-trained weights are available at https://ku-cvlab.github.io/NeMF/.

Related papers

• Semi-DETR: Semi-Supervised Object Detection with Detection Transformers [105.45018934087076]
  We analyze the DETR-based framework on semi-supervised object detection (SSOD) We present Semi-DETR, the first transformer-based end-to-end semi-supervised object detector. Our method outperforms all state-of-the-art methods by clear margins.
  arXiv  Detail & Related papers  (2023-07-16T16:32:14Z)
• Adaptive Assignment for Geometry Aware Local Feature Matching [22.818457285745733]
  detector-free feature matching approaches are currently attracting great attention thanks to their excellent performance. We introduce AdaMatcher, which accomplishes the feature correlation and co-visible area estimation through an elaborate feature interaction module. AdaMatcher then performs adaptive assignment on patch-level matching while estimating the scales between images, and finally refines the co-visible matches through scale alignment and sub-pixel regression module.
  arXiv  Detail & Related papers  (2022-07-18T08:22:18Z)
• Pipelined correlated minimum weight perfect matching of the surface code [56.01788646782563]
  We describe a pipeline approach to decoding the surface code using minimum weight perfect matching. An independent no-communication parallelizable processing stage reweights the graph according to likely correlations. A later general stage finishes the matching. We validate the new algorithm on the fully fault-tolerant toric, unrotated, and rotated surface codes.
  arXiv  Detail & Related papers  (2022-05-19T19:58:02Z)
• Autoencoding Low-Resolution MRI for Semantically Smooth Interpolation of Anisotropic MRI [1.281734910003263]
  We propose an unsupervised deep learning semantic approach that synthesizes new intermediate slices from encoded low-resolution examples. The method produces significantly better results in terms of Structural Similarity Index Measure and Peak Signal-to-Noise Ratio than a cubic B-spline approach.
  arXiv  Detail & Related papers  (2022-02-18T15:40:00Z)
• Deep Kernelized Dense Geometric Matching [14.274582421372308]
  We propose to formulate global correspondence estimation as a continuous probabilistic regression task using deep kernels. Our approach achieves significant improvements compared to the state-of-the-art on the competitive HPatches and YFCC100m benchmarks.
  arXiv  Detail & Related papers  (2022-02-01T18:58:46Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)
• GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network [176.3781969089004]
  Feature correlation layer serves as a key neural network module in computer vision problems that involve dense correspondences between image pairs. We propose GOCor, a fully differentiable dense matching module, acting as a direct replacement to the feature correlation layer. Our approach significantly outperforms the feature correlation layer for the tasks of geometric matching, optical flow, and dense semantic matching.
  arXiv  Detail & Related papers  (2020-09-16T17:33:01Z)
• Correspondence Networks with Adaptive Neighbourhood Consensus [22.013820169455812]
  We propose a convolutional neural network architecture, called adaptive neighbourhood consensus network (ANC-Net) ANC-Net can be trained end-to-end with sparse key-point annotations to handle this challenge. We thoroughly evaluate the effectiveness of our method on various benchmarks, where it substantially outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-26T17:58:09Z)
• Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning [86.45526827323954]
  Weakly-supervised semantic segmentation is a challenging task as no pixel-wise label information is provided for training. We propose an iterative algorithm to learn such pairwise relations. We show that the proposed algorithm performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-19T10:32:03Z)
• A deep learning approach for the computation of curvature in the level-set method [0.0]
  We propose a strategy to estimate the mean curvature of two-dimensional implicit in the level-set method. Our approach is based on fitting feed-forward neural networks to synthetic data sets constructed from circular immersed in uniform grids of various resolutions.
  arXiv  Detail & Related papers  (2020-02-04T00:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.