Boosting Robustness of Image Matting with Context Assembling and Strong Data Augmentation

• URL: http://arxiv.org/abs/2201.06889v1
• Date: Tue, 18 Jan 2022 11:45:17 GMT
• Title: Boosting Robustness of Image Matting with Context Assembling and Strong Data Augmentation
• Authors: Yutong Dai and Brian Price and He Zhang and Chunhua Shen
• Abstract summary: robustness to trimaps and generalization to images from different domains is still under-explored. We propose an image matting method which achieves higher robustness (RMat) via multilevel context assembling and strong data augmentation targeting matting.
• Score: 83.31087402305306
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep image matting methods have achieved increasingly better results on benchmarks (e.g., Composition-1k/alphamatting.com). However, the robustness, including robustness to trimaps and generalization to images from different domains, is still under-explored. Although some works propose to either refine the trimaps or adapt the algorithms to real-world images via extra data augmentation, none of them has taken both into consideration, not to mention the significant performance deterioration on benchmarks while using those data augmentation. To fill this gap, we propose an image matting method which achieves higher robustness (RMat) via multilevel context assembling and strong data augmentation targeting matting. Specifically, we first build a strong matting framework by modeling ample global information with transformer blocks in the encoder, and focusing on details in combination with convolution layers as well as a low-level feature assembling attention block in the decoder. Then, based on this strong baseline, we analyze current data augmentation and explore simple but effective strong data augmentation to boost the baseline model and contribute a more generalizable matting method. Compared with previous methods, the proposed method not only achieves state-of-the-art results on the Composition-1k benchmark (11% improvement on SAD and 27% improvement on Grad) with smaller model size, but also shows more robust generalization results on other benchmarks, on real-world images, and also on varying coarse-to-fine trimaps with our extensive experiments.

Related papers

• Learning Image Deraining Transformer Network with Dynamic Dual Self-Attention [46.11162082219387]
  This paper proposes an effective image deraining Transformer with dynamic dual self-attention (DDSA) Specifically, we only select the most useful similarity values based on top-k approximate calculation to achieve sparse attention. In addition, we also develop a novel spatial-enhanced feed-forward network (SEFN) to further obtain a more accurate representation for achieving high-quality derained results.
  arXiv  Detail & Related papers  (2023-08-15T13:59:47Z)
• Few-shot Image Classification based on Gradual Machine Learning [6.935034849731568]
  Few-shot image classification aims to accurately classify unlabeled images using only a few labeled samples. We propose a novel approach based on the non-i.i.d paradigm of gradual machine learning (GML) We show that the proposed approach can improve the SOTA performance by 1-5% in terms of accuracy.
  arXiv  Detail & Related papers  (2023-07-28T12:30:41Z)
• GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds [72.60362979456035]
  Masked Autoencoders (MAE) are challenging to explore in large-scale 3D point clouds. We propose a textbfGenerative textbfDecoder for MAE (GD-MAE) to automatically merges the surrounding context. We demonstrate the efficacy of the proposed method on several large-scale benchmarks: KITTI, and ONCE.
  arXiv  Detail & Related papers  (2022-12-06T14:32:55Z)
• Local Magnification for Data and Feature Augmentation [53.04028225837681]
  We propose an easy-to-implement and model-free data augmentation method called Local Magnification (LOMA) LOMA generates additional training data by randomly magnifying a local area of the image. Experiments show that our proposed LOMA, though straightforward, can be combined with standard data augmentation to significantly improve the performance on image classification and object detection.
  arXiv  Detail & Related papers  (2022-11-15T02:51:59Z)
• Feature transforms for image data augmentation [74.12025519234153]
  In image classification, many augmentation approaches utilize simple image manipulation algorithms. In this work, we build ensembles on the data level by adding images generated by combining fourteen augmentation approaches. Pretrained ResNet50 networks are finetuned on training sets that include images derived from each augmentation method.
  arXiv  Detail & Related papers  (2022-01-24T14:12:29Z)
• Contrastive Learning with Stronger Augmentations [63.42057690741711]
  We propose a general framework called Contrastive Learning with Stronger Augmentations(A) to complement current contrastive learning approaches. Here, the distribution divergence between the weakly and strongly augmented images over the representation bank is adopted to supervise the retrieval of strongly augmented queries. Experiments showed the information from the strongly augmented images can significantly boost the performance.
  arXiv  Detail & Related papers  (2021-04-15T18:40:04Z)
• FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning [64.32306537419498]
  We propose a novel learned feature-based refinement and augmentation method that produces a varied set of complex transformations. These transformations also use information from both within-class and across-class representations that we extract through clustering. We demonstrate that our method is comparable to current state of art for smaller datasets while being able to scale up to larger datasets.
  arXiv  Detail & Related papers  (2020-07-16T17:55:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.