Semantic Image Matting
- URL: http://arxiv.org/abs/2104.08201v1
- Date: Fri, 16 Apr 2021 16:21:02 GMT
- Title: Semantic Image Matting
- Authors: Yanan Sun, Chi-Keung Tang, Yu-Wing Tai
- Abstract summary: We show how to obtain better alpha mattes by incorporating into our framework semantic classification of matting regions.
Specifically, we consider and learn 20 classes of matting patterns, and propose to extend the conventional trimap to semantic trimap.
Experiments on multiple benchmarks show that our method outperforms other methods and has achieved the most competitive state-of-the-art performance.
- Score: 75.21022252141474
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Natural image matting separates the foreground from background in fractional
occupancy which can be caused by highly transparent objects, complex foreground
(e.g., net or tree), and/or objects containing very fine details (e.g., hairs).
Although conventional matting formulation can be applied to all of the above
cases, no previous work has attempted to reason the underlying causes of
matting due to various foreground semantics.
We show how to obtain better alpha mattes by incorporating into our framework
semantic classification of matting regions. Specifically, we consider and learn
20 classes of matting patterns, and propose to extend the conventional trimap
to semantic trimap. The proposed semantic trimap can be obtained automatically
through patch structure analysis within trimap regions. Meanwhile, we learn a
multi-class discriminator to regularize the alpha prediction at semantic level,
and content-sensitive weights to balance different regularization losses.
Experiments on multiple benchmarks show that our method outperforms other
methods and has achieved the most competitive state-of-the-art performance.
Finally, we contribute a large-scale Semantic Image Matting Dataset with
careful consideration of data balancing across different semantic classes.
Related papers
- DiffusionMat: Alpha Matting as Sequential Refinement Learning [87.76572845943929]
DiffusionMat is an image matting framework that employs a diffusion model for the transition from coarse to refined alpha mattes.
A correction module adjusts the output at each denoising step, ensuring that the final result is consistent with the input image's structures.
We evaluate our model across several image matting benchmarks, and the results indicate that DiffusionMat consistently outperforms existing methods.
arXiv Detail & Related papers (2023-11-22T17:16:44Z) - Matte Anything: Interactive Natural Image Matting with Segment Anything
Models [35.105593013654]
Matte Anything (MatAny) is an interactive natural image matting model that could produce high-quality alpha-matte.
We leverage vision foundation models to enhance the performance of natural image matting.
MatAny has 58.3% improvement on MSE and 40.6% improvement on SAD compared to the previous image matting methods.
arXiv Detail & Related papers (2023-06-07T03:31:39Z) - TransMatting: Enhancing Transparent Objects Matting with Transformers [4.012340049240327]
We propose a Transformer-based network, TransMatting, to model transparent objects with a big receptive field.
A small convolutional network is proposed to utilize the global feature and non-background mask to guide the multi-scale feature propagation from encoder to decoder.
We create a high-resolution matting dataset of transparent objects with small known foreground areas.
arXiv Detail & Related papers (2022-08-05T06:44:14Z) - PP-Matting: High-Accuracy Natural Image Matting [11.68134059283327]
PP-Matting is a trimap-free architecture that can achieve high-accuracy natural image matting.
Our method applies a high-resolution detail branch (HRDB) that extracts fine-grained details of the foreground.
Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask.
arXiv Detail & Related papers (2022-04-20T12:54:06Z) - Deep Automatic Natural Image Matting [82.56853587380168]
Automatic image matting (AIM) refers to estimating the soft foreground from an arbitrary natural image without any auxiliary input like trimap.
We propose a novel end-to-end matting network, which can predict a generalized trimap for any image of the above types as a unified semantic representation.
Our network trained on available composite matting datasets outperforms existing methods both objectively and subjectively.
arXiv Detail & Related papers (2021-07-15T10:29:01Z) - Smart Scribbles for Image Mating [90.18035889903909]
We propose an interactive framework, referred to as smart scribbles, to guide users to draw few scribbles on the input images.
It infers the most informative regions of an image for drawing scribbles to indicate different categories.
It then spreads these scribbles to the rest of the image via our well-designed two-phase propagation.
arXiv Detail & Related papers (2021-03-31T13:30:49Z) - Human Perception Modeling for Automatic Natural Image Matting [2.179313476241343]
Natural image matting aims to precisely separate foreground objects from background using alpha matte.
We propose an intuitively-designed trimap-free two-stage matting approach without additional annotations.
Our matting algorithm has competitive performance with current state-of-the-art methods in both trimap-free and trimap-needed aspects.
arXiv Detail & Related papers (2021-03-31T12:08:28Z) - Salient Image Matting [0.0]
We propose an image matting framework called Salient Image Matting to estimate the per-pixel opacity value of the most salient foreground in an image.
Our framework simultaneously deals with the challenge of learning a wide range of semantics and salient object types.
Our framework requires only a fraction of expensive matting data as compared to other automatic methods.
arXiv Detail & Related papers (2021-03-23T06:22:33Z) - Image Inpainting Guided by Coherence Priors of Semantics and Textures [62.92586889409379]
We introduce coherence priors between the semantics and textures which make it possible to concentrate on completing separate textures in a semantic-wise manner.
We also propose two coherence losses to constrain the consistency between the semantics and the inpainted image in terms of the overall structure and detailed textures.
arXiv Detail & Related papers (2020-12-15T02:59:37Z) - Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation [128.03739769844736]
Two neural co-attentions are incorporated into the classifier to capture cross-image semantic similarities and differences.
In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference.
Our algorithm sets new state-of-the-arts on all these settings, demonstrating well its efficacy and generalizability.
arXiv Detail & Related papers (2020-07-03T21:53:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.