Human Instance Matting via Mutual Guidance and Multi-Instance Refinement

• URL: http://arxiv.org/abs/2205.10767v1
• Date: Sun, 22 May 2022 06:56:52 GMT
• Title: Human Instance Matting via Mutual Guidance and Multi-Instance Refinement
• Authors: Yanan Sun and Chi-Keung Tang and Yu-Wing Tai
• Abstract summary: We introduce a new matting task called human instance matting (HIM) HIM requires the pertinent model to automatically predict a precise alpha matte for each human instance. Preliminary results are presented on general instance matting.
• Score: 70.06185123355249
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a new matting task called human instance matting (HIM), which requires the pertinent model to automatically predict a precise alpha matte for each human instance. Straightforward combination of closely related techniques, namely, instance segmentation, soft segmentation and human/conventional matting, will easily fail in complex cases requiring disentangling mingled colors belonging to multiple instances along hairy and thin boundary structures. To tackle these technical challenges, we propose a human instance matting framework, called InstMatt, where a novel mutual guidance strategy working in tandem with a multi-instance refinement module is used, for delineating multi-instance relationship among humans with complex and overlapping boundaries if present. A new instance matting metric called instance matting quality (IMQ) is proposed, which addresses the absence of a unified and fair means of evaluation emphasizing both instance recognition and matting quality. Finally, we construct a HIM benchmark for evaluation, which comprises of both synthetic and natural benchmark images. In addition to thorough experimental results on complex cases with multiple and overlapping human instances each has intricate boundaries, preliminary results are presented on general instance matting. Code and benchmark are available in https://github.com/nowsyn/InstMatt.

Related papers

• MaGGIe: Masked Guided Gradual Human Instance Matting [71.22209465934651]
  We propose a new framework MaGGIe, Masked Guided Gradual Human Instance Matting. It predicts alpha mattes progressively for each human instances while maintaining the computational cost, precision, and consistency.
  arXiv  Detail & Related papers  (2024-04-24T17:59:53Z)
• MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models [56.37780601189795]
  We propose a framework named MamMIL for WSI analysis. We represent each WSI as an undirected graph. To address the problem that Mamba can only process 1D sequences, we propose a topology-aware scanning mechanism.
  arXiv  Detail & Related papers  (2024-03-08T09:02:13Z)
• End-to-End Human Instance Matting [27.96723058460764]
  Human instance matting aims to estimate an alpha matte for each human instance in an image. This paper proposes a novel End-to-End Human Instance Matting (E2E-HIM) framework for simultaneous multiple instance matting. E2E-HIM outperforms the existing methods on human instance matting with 50% lower errors and 5X faster speed.
  arXiv  Detail & Related papers  (2024-03-03T13:17:10Z)
• Ins-HOI: Instance Aware Human-Object Interactions Recovery [44.02128629239429]
  We propose an end-to-end Instance-aware Human-Object Interactions recovery (Ins-HOI) framework. Ins-HOI supports instance-level reconstruction and provides reasonable and realistic invisible contact surfaces. We collect a large-scale, high-fidelity 3D scan dataset, including 5.2k high-quality scans with real-world human-chair and hand-object interactions.
  arXiv  Detail & Related papers  (2023-12-15T09:30:47Z)
• Generalizable Implicit Neural Representations via Instance Pattern Composers [40.04085054791994]
  We introduce a simple yet effective framework for generalizable implicit neural representations (INRs) Our framework is fully compatible with existing meta-learning and hypernetworks in learning to predict the weight for unseen instances. Our method achieves high performance on a wide range of domains such as an audio, image, and 3D object, while the ablation study validates our weight modulation.
  arXiv  Detail & Related papers  (2022-11-23T18:58:59Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Text Recognition in Real Scenarios with a Few Labeled Samples [55.07859517380136]
  Scene text recognition (STR) is still a hot research topic in computer vision field. This paper proposes a few-shot adversarial sequence domain adaptation (FASDA) approach to build sequence adaptation. Our approach can maximize the character-level confusion between the source domain and the target domain.
  arXiv  Detail & Related papers  (2020-06-22T13:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.