Task-Decoupled Image Inpainting Framework for Class-specific Object Remover

• URL: http://arxiv.org/abs/2410.02894v1
• Date: Thu, 3 Oct 2024 18:32:33 GMT
• Title: Task-Decoupled Image Inpainting Framework for Class-specific Object Remover
• Authors: Changsuk Oh, H. Jin Kim,
• Abstract summary: We propose a task-decoupled image inpainting framework which generates two separate inpainting models. We obtain a class-specific object remover which focuses on removing objects of a target class. Experiments show that the proposed class-specific object remover can better remove target class objects than object removers based on image inpainting networks.
• Score: 26.81218265405809
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object removal refers to the process of erasing designated objects from an image while preserving the overall appearance. Existing works on object removal erase removal targets using image inpainting networks. However, image inpainting networks often generate unsatisfactory removal results. In this work, we find that the current training approach which encourages a single image inpainting model to handle both object removal and restoration tasks is one of the reasons behind such unsatisfactory result. Based on this finding, we propose a task-decoupled image inpainting framework which generates two separate inpainting models: an object restorer for object restoration tasks and an object remover for object removal tasks. We train the object restorer with the masks that partially cover the removal targets. Then, the proposed framework makes an object restorer to generate a guidance for training the object remover. Using the proposed framework, we obtain a class-specific object remover which focuses on removing objects of a target class, aiming to better erase target class objects than general object removers. We also introduce a data curation method that encompasses the image selection and mask generation approaches used to produce training data for the proposed class-specific object remover. Using the proposed curation method, we can simulate the scenarios where an object remover is trained on the data with object removal ground truth images. Experiments on multiple datasets show that the proposed class-specific object remover can better remove target class objects than object removers based on image inpainting networks.

Related papers

• Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model [81.96954332787655]
  We introduce Diffree, a Text-to-Image (T2I) model that facilitates text-guided object addition with only text control. In experiments, Diffree adds new objects with a high success rate while maintaining background consistency, spatial, and object relevance and quality.
  arXiv  Detail & Related papers  (2024-07-24T03:58:58Z)
• Retrieval Robust to Object Motion Blur [54.34823913494456]
  We propose a method for object retrieval in images that are affected by motion blur. We present the first large-scale datasets for blurred object retrieval. Our method outperforms state-of-the-art retrieval methods on the new blur-retrieval datasets.
  arXiv  Detail & Related papers  (2024-04-27T23:22:39Z)
• Object Remover Performance Evaluation Methods using Class-wise Object Removal Images [29.28862692741578]
  The performance of an object remover is quantitatively evaluated by measuring the quality of object removal results. We create a dataset with object removal ground truth and compare the evaluations made by the current methods using original images. We propose new evaluation methods tailored to gauge the performance of an object remover.
  arXiv  Detail & Related papers  (2024-04-17T06:40:47Z)
• AURA : Automatic Mask Generator using Randomized Input Sampling for Object Removal [26.81218265405809]
  In this paper, we focus on generating the input mask to better remove objects using the off-the-shelf image inpainting network. We propose an automatic mask generator inspired by the explainable AI (XAI) method, whose output can better remove objects than a semantic segmentation mask. Experiments confirm that our method shows better performance in removing target class objects than the masks generated from the semantic segmentation maps.
  arXiv  Detail & Related papers  (2023-05-13T07:51:35Z)
• Inst-Inpaint: Instructing to Remove Objects with Diffusion Models [18.30057229657246]
  In this work, we are interested in an image inpainting algorithm that estimates which object to be removed based on natural language input and removes it, simultaneously. We present a novel inpainting framework, Inst-Inpaint, that can remove objects from images based on the instructions given as text prompts.
  arXiv  Detail & Related papers  (2023-04-06T17:29:50Z)
• Image Segmentation-based Unsupervised Multiple Objects Discovery [1.7674345486888503]
  Unsupervised object discovery aims to localize objects in images. We propose a fully unsupervised, bottom-up approach, for multiple objects discovery. We provide state-of-the-art results for both unsupervised class-agnostic object detection and unsupervised image segmentation.
  arXiv  Detail & Related papers  (2022-12-20T09:48:24Z)
• Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators [97.12135238534628]
  We propose a learning paradigm that consists of semantic discriminators and object-level discriminators for improving the generation of complex semantics and objects. Specifically, the semantic discriminators leverage pretrained visual features to improve the realism of the generated visual concepts. Our proposed scheme significantly improves the generation quality and achieves state-of-the-art results on various tasks.
  arXiv  Detail & Related papers  (2022-12-13T01:36:56Z)
• Shape-guided Object Inpainting [84.18768707298105]
  This work studies a new image inpainting task, i.e. shape-guided object inpainting. We propose a new data preparation method and a novel Contextual Object Generator (CogNet) for the object inpainting task. Experiments demonstrate that the proposed method can generate realistic objects that fit the context in terms of both visual appearance and semantic meanings.
  arXiv  Detail & Related papers  (2022-04-16T17:19:11Z)
• A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection [56.82077636126353]
  We take advantage of object-centric images to improve object detection in scene-centric images. We present a simple yet surprisingly effective framework to do so. Our approach can improve the object detection (and instance segmentation) accuracy of rare objects by 50% (and 33%) relatively.
  arXiv  Detail & Related papers  (2021-02-17T17:27:21Z)
• Disassembling Object Representations without Labels [75.2215716328001]
  We study a new representation-learning task, which we termed as disassembling object representations. Disassembling enables category-specific modularity in the learned representations. We propose an unsupervised approach to achieving disassembling, named Unsupervised Disassembling Object Representation (UDOR)
  arXiv  Detail & Related papers  (2020-04-03T08:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.