Towards Label-Efficient Human Matting: A Simple Baseline for Weakly Semi-Supervised Trimap-Free Human Matting

• URL: http://arxiv.org/abs/2404.00921v1
• Date: Mon, 1 Apr 2024 04:53:06 GMT
• Title: Towards Label-Efficient Human Matting: A Simple Baseline for Weakly Semi-Supervised Trimap-Free Human Matting
• Authors: Beomyoung Kim, Myeong Yeon Yi, Joonsang Yu, Young Joon Yoo, Sung Ju Hwang,
• Abstract summary: We introduce a new learning paradigm, weakly semi-supervised human matting (WSSHM) WSSHM uses a small amount of expensive matte labels and a large amount of budget-friendly segmentation labels to save the annotation cost and resolve the domain generalization problem. Our training method is also easily applicable to real-time models, achieving competitive accuracy with breakneck inference speed.
• Score: 50.99997483069828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a new practical training method for human matting, which demands delicate pixel-level human region identification and significantly laborious annotations. To reduce the annotation cost, most existing matting approaches often rely on image synthesis to augment the dataset. However, the unnaturalness of synthesized training images brings in a new domain generalization challenge for natural images. To address this challenge, we introduce a new learning paradigm, weakly semi-supervised human matting (WSSHM), which leverages a small amount of expensive matte labels and a large amount of budget-friendly segmentation labels, to save the annotation cost and resolve the domain generalization problem. To achieve the goal of WSSHM, we propose a simple and effective training method, named Matte Label Blending (MLB), that selectively guides only the beneficial knowledge of the segmentation and matte data to the matting model. Extensive experiments with our detailed analysis demonstrate our method can substantially improve the robustness of the matting model using a few matte data and numerous segmentation data. Our training method is also easily applicable to real-time models, achieving competitive accuracy with breakneck inference speed (328 FPS on NVIDIA V100 GPU). The implementation code is available at \url{https://github.com/clovaai/WSSHM}.

Related papers

• Diffusion for Natural Image Matting [93.86689168212241]
  We present DiffMatte, a solution designed to overcome the challenges of image matting. First, DiffMatte decouples the decoder from the intricately coupled matting network design, involving only one lightweight decoder in the iterations of the diffusion process. Second, we employ a self-aligned training strategy with uniform time intervals, ensuring a consistent noise sampling between training and inference across the entire time domain.
  arXiv  Detail & Related papers  (2023-12-10T15:28:56Z)
• SGM-Net: Semantic Guided Matting Net [5.126872642595207]
  We propose a module to generate foreground probability map and add it to MODNet to obtain Semantic Guided Matting Net (SGM-Net) Under the condition of only one image, we can realize the human matting task.
  arXiv  Detail & Related papers  (2022-08-16T01:58:25Z)
• Bi-level Alignment for Cross-Domain Crowd Counting [113.78303285148041]
  Current methods rely on external data for training an auxiliary task or apply an expensive coarse-to-fine estimation. We develop a new adversarial learning based method, which is simple and efficient to apply. We evaluate our approach on five real-world crowd counting benchmarks, where we outperform existing approaches by a large margin.
  arXiv  Detail & Related papers  (2022-05-12T02:23:25Z)
• Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images [0.2538209532048867]
  We propose a semi-weakly supervised contrastive learning framework for representation learning using semi-weakly annotated images. We empirically validate the transfer learning performance of SWCL on seven public retinal fundus datasets.
  arXiv  Detail & Related papers  (2021-08-04T15:50:09Z)
• Towards Single Stage Weakly Supervised Semantic Segmentation [2.28438857884398]
  We present a single-stage approach to weakly supervised semantic segmentation. We use point annotations to generate reliable, on-the-fly pseudo-masks. We significantly outperform other SOTA WSSS methods on recent real-world datasets.
  arXiv  Detail & Related papers  (2021-06-18T18:34:50Z)
• Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets [90.61266099147053]
  We investigate efficient annotation strategies for collecting multi-class classification labels for a large collection of images. We propose modifications and best practices aimed at minimizing human labeling effort. Simulated experiments on a 125k image subset of the ImageNet100 show that it can be annotated to 80% top-1 accuracy with 0.35 annotations per image on average.
  arXiv  Detail & Related papers  (2021-04-26T16:29:32Z)
• Leveraging Self-Supervision for Cross-Domain Crowd Counting [71.75102529797549]
  State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density. We train our network to recognize upside-down real images from regular ones and incorporate into it the ability to predict its own uncertainty. This yields an algorithm that consistently outperforms state-of-the-art cross-domain crowd counting ones without any extra computation at inference time.
  arXiv  Detail & Related papers  (2021-03-30T12:37:55Z)
• Learning to Sample the Most Useful Training Patches from Images [11.219920058662698]
  We present a data-driven approach called PatchNet that learns to select the most useful patches from an image to construct a new training set. We show that our simple idea automatically selects informative samples out from a large-scale dataset, leading to a surprising 2.35dB generalisation gain in terms of PSNR.
  arXiv  Detail & Related papers  (2020-11-24T14:06:50Z)
• Improving Semantic Segmentation via Self-Training [75.07114899941095]
  We show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm. We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data. Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets.
  arXiv  Detail & Related papers  (2020-04-30T17:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.