Robust Human Matting via Semantic Guidance

• URL: http://arxiv.org/abs/2210.05210v1
• Date: Tue, 11 Oct 2022 07:25:33 GMT
• Title: Robust Human Matting via Semantic Guidance
• Authors: Xiangguang Chen, Ye Zhu, Yu Li, Bingtao Fu, Lei Sun, Ying Shan and Shan Liu
• Abstract summary: We develop a fast yet accurate human matting framework, named Semantic Guided Human Matting (SGHM) It builds on a semantic human segmentation network and introduces a light-weight matting module with only marginal computational cost. Our experiments show that trained with merely 200 matting images, our method can generalize well to real-world datasets.
• Score: 35.374012964806745
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic human matting is highly desired for many real applications. We investigate recent human matting methods and show that common bad cases happen when semantic human segmentation fails. This indicates that semantic understanding is crucial for robust human matting. From this, we develop a fast yet accurate human matting framework, named Semantic Guided Human Matting (SGHM). It builds on a semantic human segmentation network and introduces a light-weight matting module with only marginal computational cost. Unlike previous works, our framework is data efficient, which requires a small amount of matting ground-truth to learn to estimate high quality object mattes. Our experiments show that trained with merely 200 matting images, our method can generalize well to real-world datasets, and outperform recent methods on multiple benchmarks, while remaining efficient. Considering the unbearable labeling cost of matting data and widely available segmentation data, our method becomes a practical and effective solution for the task of human matting. Source code is available at https://github.com/cxgincsu/SemanticGuidedHumanMatting.

Related papers

• Towards Label-Efficient Human Matting: A Simple Baseline for Weakly Semi-Supervised Trimap-Free Human Matting [50.99997483069828]
  We introduce a new learning paradigm, weakly semi-supervised human matting (WSSHM) WSSHM uses a small amount of expensive matte labels and a large amount of budget-friendly segmentation labels to save the annotation cost and resolve the domain generalization problem. Our training method is also easily applicable to real-time models, achieving competitive accuracy with breakneck inference speed.
  arXiv  Detail & Related papers  (2024-04-01T04:53:06Z)
• Dexterous Functional Grasping [39.15442658671798]
  This paper combines the best of both worlds to accomplish functional grasping for in-the-wild objects. We propose a novel application of eigengrasps to reduce the search space of RL using a small amount of human data. We find that eigengrasp action space beats baselines in simulation and outperforms hardcoded grasping in real and matches or outperforms a trained human teleoperator.
  arXiv  Detail & Related papers  (2023-12-05T18:59:23Z)
• Learning Human Action Recognition Representations Without Real Humans [66.61527869763819]
  We present a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model. We then evaluate the transferability of the representation learned on this data to a diverse set of downstream action recognition benchmarks. Our approach outperforms previous baselines by up to 5%.
  arXiv  Detail & Related papers  (2023-11-10T18:38:14Z)
• SGM-Net: Semantic Guided Matting Net [5.126872642595207]
  We propose a module to generate foreground probability map and add it to MODNet to obtain Semantic Guided Matting Net (SGM-Net) Under the condition of only one image, we can realize the human matting task.
  arXiv  Detail & Related papers  (2022-08-16T01:58:25Z)
• Bi-level Alignment for Cross-Domain Crowd Counting [113.78303285148041]
  Current methods rely on external data for training an auxiliary task or apply an expensive coarse-to-fine estimation. We develop a new adversarial learning based method, which is simple and efficient to apply. We evaluate our approach on five real-world crowd counting benchmarks, where we outperform existing approaches by a large margin.
  arXiv  Detail & Related papers  (2022-05-12T02:23:25Z)
• Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing [131.97475877877608]
  A new bottom-up regime is proposed to learn category-level human semantic segmentation and multi-person pose estimation in a joint and end-to-end manner. It is a compact, efficient and powerful framework that exploits structural information over different human granularities. Experiments on three instance-aware human datasets show that our model outperforms other bottom-up alternatives with much more efficient inference.
  arXiv  Detail & Related papers  (2021-03-08T06:55:00Z)
• Boosting Semantic Human Matting with Coarse Annotations [66.8725980604434]
  coarse annotated human dataset is much easier to acquire and collect from the public dataset. A matting refinement network takes in the unified mask and the input image to predict the final alpha matte.
  arXiv  Detail & Related papers  (2020-04-10T09:11:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.