Alpha Matte Generation from Single Input for Portrait Matting
- URL: http://arxiv.org/abs/2106.03210v1
- Date: Sun, 6 Jun 2021 18:53:42 GMT
- Title: Alpha Matte Generation from Single Input for Portrait Matting
- Authors: Dogucan Yaman and Haz{\i}m Kemal Ekenel and Alexander Waibel
- Abstract summary: The goal is to predict an alpha matte that identifies the effect of each pixel on the foreground subject.
Traditional approaches and most of the existing works utilized an additional input, e.g., trimap, background image, to predict alpha matte.
We introduce an additional input-free approach to perform portrait matting using Generative Adversarial Nets (GANs)
- Score: 79.62140902232628
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Portrait matting is an important research problem with a wide range of
applications, such as video conference app, image/video editing, and
post-production. The goal is to predict an alpha matte that identifies the
effect of each pixel on the foreground subject. Traditional approaches and most
of the existing works utilized an additional input, e.g., trimap, background
image, to predict alpha matte. However, providing additional input is not
always practical. Besides, models are too sensitive to these additional inputs.
In this paper, we introduce an additional input-free approach to perform
portrait matting using Generative Adversarial Nets (GANs). We divide the main
task into two subtasks. For this, we propose a segmentation network for the
person segmentation and the alpha generation network for alpha matte
prediction. While the segmentation network takes an input image and produces a
coarse segmentation map, the alpha generation network utilizes the same input
image as well as a coarse segmentation map that is produced by the segmentation
network to predict the alpha matte. Besides, we present a segmentation encoding
block to downsample the coarse segmentation map and provide feature
representation to the residual block. Furthermore, we propose border loss to
penalize only the borders of the subject separately which is more likely to be
challenging and we also adapt perceptual loss for portrait matting. To train
the proposed system, we combine two different popular training datasets to
improve the amount of data as well as diversity to address domain shift
problems in the inference time. We tested our model on three different
benchmark datasets, namely Adobe Image Matting dataset, Portrait Matting
dataset, and Distinctions dataset. The proposed method outperformed the MODNet
method that also takes a single input.
Related papers
- Training Matting Models without Alpha Labels [22.249204770416927]
This work explores using rough annotations such as trimaps coarsely indicating the foreground/background as supervision.
We present that the cooperation between learned semantics from indicated known regions and proper assumed matting rules can help infer alpha values at transition areas.
Experiments on AM-2K and P3M-10K dataset show that our paradigm achieves comparable performance with the fine-label-supervised baseline.
arXiv Detail & Related papers (2024-08-20T04:34:06Z) - Facial Wrinkle Segmentation for Cosmetic Dermatology: Pretraining with Texture Map-Based Weak Supervision [0.053801353100098995]
We build and release the first public facial wrinkle dataset, 'FFHQ-Wrinkle', an extension of the NVIDIA FFHQ dataset.
It includes 1,000 images with human labels and 50,000 images with automatically generated weak labels.
This dataset could serve as a foundation for the research community to develop advanced wrinkle detection algorithms.
arXiv Detail & Related papers (2024-08-19T14:54:12Z) - DiffusionMat: Alpha Matting as Sequential Refinement Learning [87.76572845943929]
DiffusionMat is an image matting framework that employs a diffusion model for the transition from coarse to refined alpha mattes.
A correction module adjusts the output at each denoising step, ensuring that the final result is consistent with the input image's structures.
We evaluate our model across several image matting benchmarks, and the results indicate that DiffusionMat consistently outperforms existing methods.
arXiv Detail & Related papers (2023-11-22T17:16:44Z) - Lidar Annotation Is All You Need [0.0]
This paper aims to improve the efficiency of image segmentation using a convolutional neural network in a multi-sensor setup.
The key innovation of our approach is the masked loss, addressing sparse ground-truth masks from point clouds.
Experimental validation of the approach on benchmark datasets shows comparable performance to a high-quality image segmentation model.
arXiv Detail & Related papers (2023-11-08T15:55:18Z) - Scrape, Cut, Paste and Learn: Automated Dataset Generation Applied to
Parcel Logistics [58.720142291102135]
We present a fully automated pipeline to generate a synthetic dataset for instance segmentation in four steps.
We first scrape images for the objects of interest from popular image search engines.
We compare three different methods for image selection: Object-agnostic pre-processing, manual image selection and CNN-based image selection.
arXiv Detail & Related papers (2022-10-18T12:49:04Z) - Attention-guided Temporal Coherent Video Object Matting [78.82835351423383]
We propose a novel deep learning-based object matting method that can achieve temporally coherent matting results.
Its key component is an attention-based temporal aggregation module that maximizes image matting networks' strength.
We show how to effectively solve the trimap generation problem by fine-tuning a state-of-the-art video object segmentation network.
arXiv Detail & Related papers (2021-05-24T17:34:57Z) - Semantic Segmentation with Generative Models: Semi-Supervised Learning
and Strong Out-of-Domain Generalization [112.68171734288237]
We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels.
We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images.
We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
arXiv Detail & Related papers (2021-04-12T21:41:25Z) - Boosting Semantic Human Matting with Coarse Annotations [66.8725980604434]
coarse annotated human dataset is much easier to acquire and collect from the public dataset.
A matting refinement network takes in the unified mask and the input image to predict the final alpha matte.
arXiv Detail & Related papers (2020-04-10T09:11:02Z) - $F$, $B$, Alpha Matting [0.0]
We propose a low-cost modification to alpha matting networks to also predict the foreground and background colours.
Our method achieves the state of the art performance on the Adobe Composition-1k dataset for alpha matte and composite colour quality.
arXiv Detail & Related papers (2020-03-17T13:27:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.