Deep Image Matting with Flexible Guidance Input

• URL: http://arxiv.org/abs/2110.10898v1
• Date: Thu, 21 Oct 2021 04:59:27 GMT
• Title: Deep Image Matting with Flexible Guidance Input
• Authors: Hang Cheng, Shugong Xu, Xiufeng Jiang, Rongrong Wang
• Abstract summary: We propose a matting method that use Flexible Guidance Input as user hint. Our method can achieve state-of-the-art results comparing with existing trimap-based and trimap-free methods.
• Score: 16.651948566049846
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image matting is an important computer vision problem. Many existing matting methods require a hand-made trimap to provide auxiliary information, which is very expensive and limits the real world usage. Recently, some trimap-free methods have been proposed, which completely get rid of any user input. However, their performance lag far behind trimap-based methods due to the lack of guidance information. In this paper, we propose a matting method that use Flexible Guidance Input as user hint, which means our method can use trimap, scribblemap or clickmap as guidance information or even work without any guidance input. To achieve this, we propose Progressive Trimap Deformation(PTD) scheme that gradually shrink the area of the foreground and background of the trimap with the training step increases and finally become a scribblemap. To make our network robust to any user scribble and click, we randomly sample points on foreground and background and perform curve fitting. Moreover, we propose Semantic Fusion Module(SFM) which utilize the Feature Pyramid Enhancement Module(FPEM) and Joint Pyramid Upsampling(JPU) in matting task for the first time. The experiments show that our method can achieve state-of-the-art results comparing with existing trimap-based and trimap-free methods.

Related papers

• Learning Trimaps via Clicks for Image Matting [103.6578944248185]
  We introduce Click2Trimap, an interactive model capable of predicting high-quality trimaps and alpha mattes with minimal user click inputs. In the user study, Click2Trimap achieves high-quality trimap and matting predictions in just an average of 5 seconds per image.
  arXiv  Detail & Related papers  (2024-03-30T12:10:34Z)
• Neural Semantic Surface Maps [52.61017226479506]
  We present an automated technique for computing a map between two genus-zero shapes, which matches semantically corresponding regions to one another. Our approach can generate semantic surface-to-surface maps, eliminating manual annotations or any 3D training data requirement.
  arXiv  Detail & Related papers  (2023-09-09T16:21:56Z)
• TMO: Textured Mesh Acquisition of Objects with a Mobile Device by using Differentiable Rendering [54.35405028643051]
  We present a new pipeline for acquiring a textured mesh in the wild with a single smartphone. Our method first introduces an RGBD-aided structure from motion, which can yield filtered depth maps. We adopt the neural implicit surface reconstruction method, which allows for high-quality mesh.
  arXiv  Detail & Related papers  (2023-03-27T10:07:52Z)
• PP-Matting: High-Accuracy Natural Image Matting [11.68134059283327]
  PP-Matting is a trimap-free architecture that can achieve high-accuracy natural image matting. Our method applies a high-resolution detail branch (HRDB) that extracts fine-grained details of the foreground. Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask.
  arXiv  Detail & Related papers  (2022-04-20T12:54:06Z)
• Polygonal Point Set Tracking [50.445151155209246]
  We propose a novel learning-based polygonal point set tracking method. Our goal is to track corresponding points on the target contour. We present visual-effects applications of our method on part distortion and text mapping.
  arXiv  Detail & Related papers  (2021-05-30T17:12:36Z)
• Human Perception Modeling for Automatic Natural Image Matting [2.179313476241343]
  Natural image matting aims to precisely separate foreground objects from background using alpha matte. We propose an intuitively-designed trimap-free two-stage matting approach without additional annotations. Our matting algorithm has competitive performance with current state-of-the-art methods in both trimap-free and trimap-needed aspects.
  arXiv  Detail & Related papers  (2021-03-31T12:08:28Z)
• Salient Image Matting [0.0]
  We propose an image matting framework called Salient Image Matting to estimate the per-pixel opacity value of the most salient foreground in an image. Our framework simultaneously deals with the challenge of learning a wide range of semantics and salient object types. Our framework requires only a fraction of expensive matting data as compared to other automatic methods.
  arXiv  Detail & Related papers  (2021-03-23T06:22:33Z)
• Improved Image Matting via Real-time User Clicks and Uncertainty Estimation [87.84632514927098]
  This paper proposes an improved deep image matting framework which is trimap-free and only needs several user click interactions to eliminate the ambiguity. We introduce a new uncertainty estimation module that can predict which parts need polishing and a following local refinement module. Results show that our method performs better than existing trimap-free methods and comparably to state-of-the-art trimap-based methods with minimal user effort.
  arXiv  Detail & Related papers  (2020-12-15T14:32:36Z)
• Towards Reading Beyond Faces for Sparsity-Aware 4D Affect Recognition [55.15661254072032]
  We present a sparsity-aware deep network for automatic 4D facial expression recognition (FER) We first propose a novel augmentation method to combat the data limitation problem for deep learning. We then present a sparsity-aware deep network to compute the sparse representations of convolutional features over multi-views.
  arXiv  Detail & Related papers  (2020-02-08T13:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.