NeuralMarker: A Framework for Learning General Marker Correspondence

• URL: http://arxiv.org/abs/2209.08896v1
• Date: Mon, 19 Sep 2022 10:04:38 GMT
• Title: NeuralMarker: A Framework for Learning General Marker Correspondence
• Authors: Zhaoyang Huang, Xiaokun Pan, Weihong Pan, Weikang Bian, Yan Xu, Ka Chun Cheung, Guofeng Zhang, Hongsheng Li
• Abstract summary: We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker. We propose a novel framework NeuralMarker, training a neural network estimating dense marker correspondences under various challenging conditions. We show that NeuralMarker significantly outperforms previous methods and enables new interesting applications, including Augmented Reality (AR) and video editing.
• Score: 25.822657926255573
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker. Conventionally, this problem is addressed by fitting a homography model based on sparse feature matching. However, they are only able to handle plane-like markers and the sparse features do not sufficiently utilize appearance information. In this paper, we propose a novel framework NeuralMarker, training a neural network estimating dense marker correspondences under various challenging conditions, such as marker deformation, harsh lighting, etc. Besides, we also propose a novel marker correspondence evaluation method circumstancing annotations on real marker-image pairs and create a new benchmark. We show that NeuralMarker significantly outperforms previous methods and enables new interesting applications, including Augmented Reality (AR) and video editing.

Related papers

• Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentation [1.878433493707693]
  Recent progress in interactive point prompt based Image allows to significantly reduce the manual effort to obtain high quality semantic labels. We propose a novel unsupervised and training-free approach based solely on the self-attention of Stable Diffusion.
  arXiv  Detail & Related papers  (2024-11-15T18:29:59Z)
• New Efficient Visual OILU Markers [0.5120567378386615]
  We will exploit basic patterns to develop new efficient visual markers. The proposed markers allow producing rich panel of unique identifiers. The robustness of the markers against acquisition and geometric distortions is validated.
  arXiv  Detail & Related papers  (2024-04-12T13:55:05Z)
• Guided Interpretable Facial Expression Recognition via Spatial Action Unit Cues [55.97779732051921]
  A new learning strategy is proposed to explicitly incorporate au cues into classifier training. We show that our strategy can improve layer-wise interpretability without degrading classification performance.
  arXiv  Detail & Related papers  (2024-02-01T02:13:49Z)
• OsmLocator: locating overlapping scatter marks with a non-training generative perspective [48.50108853199417]
  Locating overlapping marks faces many difficulties such as no texture, less contextual information, hallow shape and tiny size. Here, we formulate it as a optimization problem on clustering-based re-visualization from a non-training generative perspective. We especially built a dataset named 2023 containing hundreds of scatter images with different markers and various levels of overlapping severity, and tested the proposed method and compared it to existing methods.
  arXiv  Detail & Related papers  (2023-12-18T12:39:48Z)
• A Locality-based Neural Solver for Optical Motion Capture [37.28597049192196]
  Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes. We show that our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error.
  arXiv  Detail & Related papers  (2023-09-01T12:40:17Z)
• MAPLE: Masked Pseudo-Labeling autoEncoder for Semi-supervised Point Cloud Action Recognition [160.49403075559158]
  We propose a Masked Pseudo-Labeling autoEncoder (textbfMAPLE) framework for point cloud action recognition. In particular, we design a novel and efficient textbfDecoupled textbfspatial-textbftemporal TranstextbfFormer (textbfDestFormer) as the backbone of MAPLE. MAPLE achieves superior results on three public benchmarks and outperforms the state-of-the-art method by 8.08% accuracy on the MSR-Action3
  arXiv  Detail & Related papers  (2022-09-01T12:32:40Z)
• Learning Semantic Correspondence with Sparse Annotations [66.37298464505261]
  Finding dense semantic correspondence is a fundamental problem in computer vision. We propose a teacher-student learning paradigm for generating dense pseudo-labels. We also develop two novel strategies for denoising pseudo-labels.
  arXiv  Detail & Related papers  (2022-08-15T02:24:18Z)
• DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers [27.135078472097895]
  Existing detection methods assume that markers are printed on ideally planar surfaces. A fiducial marker generator creates a set of free-form color patterns to encode significantly large-scale information. A differentiable image simulator creates a training dataset of photorealistic scene images with the deformed markers. A trained marker detector seeks the regions of interest and recognizes multiple marker patterns simultaneously.
  arXiv  Detail & Related papers  (2022-06-16T09:29:26Z)
• SparseAlign: A Super-Resolution Algorithm for Automatic Marker Localization and Deformation Estimation in Cryo-Electron Tomography [0.17398560678845074]
  We extend a grid-free super-resolution algorithm first proposed in the context of single-molecule localization microscopy. Our approach does not require labelled marker locations; instead, we use an image-based loss where we compare the forward projection of markers with the observed data. We show that our approach automatically finds markers and reliably estimates sample deformation without labelled marker data.
  arXiv  Detail & Related papers  (2022-01-21T14:03:32Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• Data Augmentation for Object Detection via Differentiable Neural Rendering [71.00447761415388]
  It is challenging to train a robust object detector when annotated data is scarce. Existing approaches to tackle this problem include semi-supervised learning that interpolates labeled data from unlabeled data. We introduce an offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
  arXiv  Detail & Related papers  (2021-03-04T06:31:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.