Mask-Guided Image Person Removal with Data Synthesis

• URL: http://arxiv.org/abs/2209.14890v1
• Date: Thu, 29 Sep 2022 15:58:17 GMT
• Title: Mask-Guided Image Person Removal with Data Synthesis
• Authors: Yunliang Jiang, Chenyang Gu, Zhenfeng Xue, Xiongtao Zhang, Yong Liu
• Abstract summary: We propose a novel idea to tackle these problems from the perspective of data synthesis. Concerning the lack of dedicated dataset for image person removal, two dataset production methods are proposed to automatically generate images, masks and ground truths respectively. A learning framework similar to local image degradation is proposed so that the masks can be used to guide the feature extraction process and more texture information can be gathered for final prediction.
• Score: 11.207512995742999
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As a special case of common object removal, image person removal is playing an increasingly important role in social media and criminal investigation domains. Due to the integrity of person area and the complexity of human posture, person removal has its own dilemmas. In this paper, we propose a novel idea to tackle these problems from the perspective of data synthesis. Concerning the lack of dedicated dataset for image person removal, two dataset production methods are proposed to automatically generate images, masks and ground truths respectively. Then, a learning framework similar to local image degradation is proposed so that the masks can be used to guide the feature extraction process and more texture information can be gathered for final prediction. A coarse-to-fine training strategy is further applied to refine the details. The data synthesis and learning framework combine well with each other. Experimental results verify the effectiveness of our method quantitatively and qualitatively, and the trained network proves to have good generalization ability either on real or synthetic images.

Related papers

• Image Captions are Natural Prompts for Text-to-Image Models [70.30915140413383]
  We analyze the relationship between the training effect of synthetic data and the synthetic data distribution induced by prompts. We propose a simple yet effective method that prompts text-to-image generative models to synthesize more informative and diverse training data. Our method significantly improves the performance of models trained on synthetic training data.
  arXiv  Detail & Related papers  (2023-07-17T14:38:11Z)
• Global Context-Aware Person Image Generation [24.317541784957285]
  We propose a data-driven approach for context-aware person image generation. In our method, the position, scale, and appearance of the generated person are semantically conditioned on the existing persons in the scene.
  arXiv  Detail & Related papers  (2023-02-28T16:34:55Z)
• Synthetic Data for Object Classification in Industrial Applications [53.180678723280145]
  In object classification, capturing a large number of images per object and in different conditions is not always possible. This work explores the creation of artificial images using a game engine to cope with limited data in the training dataset.
  arXiv  Detail & Related papers  (2022-12-09T11:43:04Z)
• Person Image Synthesis via Denoising Diffusion Model [116.34633988927429]
  We show how denoising diffusion models can be applied for high-fidelity person image synthesis. Our results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios.
  arXiv  Detail & Related papers  (2022-11-22T18:59:50Z)
• Pose Guided Human Image Synthesis with Partially Decoupled GAN [25.800174118151638]
  Pose Guided Human Image Synthesis (PGHIS) is a challenging task of transforming a human image from the reference pose to a target pose. We propose a method by decoupling the human body into several parts to guide the synthesis of a realistic image of the person. In addition, we design a multi-head attention-based module for PGHIS.
  arXiv  Detail & Related papers  (2022-10-07T15:31:37Z)
• Cut-Paste Consistency Learning for Semi-Supervised Lesion Segmentation [0.20305676256390934]
  Semi-supervised learning has the potential to improve the data-efficiency of training data-hungry deep neural networks. We present a simple semi-supervised learning method for lesion segmentation tasks based on the ideas of cut-paste augmentation and consistency regularization.
  arXiv  Detail & Related papers  (2022-10-01T04:43:54Z)
• Synthetic-to-Real Domain Adaptation using Contrastive Unpaired Translation [28.19031441659854]
  We propose a multi-step method to obtain training data without manual annotation effort. From 3D object meshes, we generate images using a modern synthesis pipeline. We utilize a state-of-the-art image-to-image translation method to adapt the synthetic images to the real domain.
  arXiv  Detail & Related papers  (2022-03-17T17:13:23Z)
• Bridging Composite and Real: Towards End-to-end Deep Image Matting [88.79857806542006]
  We study the roles of semantics and details for image matting. We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders. Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-30T10:57:13Z)
• Boosting Semantic Human Matting with Coarse Annotations [66.8725980604434]
  coarse annotated human dataset is much easier to acquire and collect from the public dataset. A matting refinement network takes in the unified mask and the input image to predict the final alpha matte.
  arXiv  Detail & Related papers  (2020-04-10T09:11:02Z)
• Learning Inverse Rendering of Faces from Real-world Videos [52.313931830408386]
  Existing methods decompose a face image into three components (albedo, normal, and illumination) by supervised training on synthetic data. We propose a weakly supervised training approach to train our model on real face videos, based on the assumption of consistency of albedo and normal. Our network is trained on both real and synthetic data, benefiting from both.
  arXiv  Detail & Related papers  (2020-03-26T17:26:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.