Making Images Real Again: A Comprehensive Survey on Deep Image Composition

• URL: http://arxiv.org/abs/2106.14490v6
• Date: Mon, 20 Jan 2025 04:45:01 GMT
• Title: Making Images Real Again: A Comprehensive Survey on Deep Image Composition
• Authors: Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, Liqing Zhang,
• Abstract summary: Image composition task could be decomposed into multiple sub-tasks. Object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background.
• Score: 34.09380539557308
• License:
• Abstract: As a common image editing operation, image composition (object insertion) aims to combine the foreground from one image and another background image, resulting in a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which includes appearance inconsistency (e.g., incompatible illumination), geometry inconsistency (e.g., unreasonable size), and semantic inconsistency (e.g., mismatched semantic context). Image composition task could be decomposed into multiple sub-tasks, in which each sub-task targets at one or more issues. Specifically, object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background. Image harmonization aims to adjust the illumination statistics of foreground. Shadow (resp., reflection) generation aims to generate plausible shadow (resp., reflection) for the foreground. These sub-tasks can be executed sequentially or parallelly to acquire realistic composite images. To the best of our knowledge, there is no previous survey on image composition (object insertion). In this paper, we conduct comprehensive survey over the sub-tasks and combinatorial task of image composition (object insertion). For each one, we summarize the existing methods, available datasets, and common evaluation metrics. We have also contributed the first image composition toolbox libcom, which assembles 10+ image composition related functions.

Related papers

• DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Composition [7.693732944239458]
  DepGAN is a Generative Adversarial Network that utilizes depth maps and alpha channels to rectify inaccurate occlusions. Central to our network is a novel loss function called Depth Aware Loss which quantifies the pixel wise depth difference. We enhance our network's learning process by utilizing opacity data, enabling it to effectively manage compositions involving transparent and semi-transparent objects.
  arXiv  Detail & Related papers  (2024-07-16T16:18:40Z)
• Repositioning the Subject within Image [78.8467524191102]
  We introduce an innovative dynamic manipulation task, subject repositioning. This task involves relocating a user-specified subject to a desired position while preserving the image's fidelity. Our research reveals that the fundamental sub-tasks of subject repositioning can be effectively reformulated as a unified, prompt-guided inpainting task.
  arXiv  Detail & Related papers  (2024-01-30T10:04:49Z)
• AMICO: Amodal Instance Composition [40.03865667370814]
  Image composition aims to blend multiple objects to form a harmonized image. We present Amodal Instance Composition for blending imperfect objects onto a target image. Our results show state-of-the-art performance on public COCOA and KINS benchmarks.
  arXiv  Detail & Related papers  (2022-10-11T23:23:14Z)
• Blind Image Decomposition [53.760745569495825]
  We present Blind Image Decomposition (BID), which requires separating a superimposed image into constituent underlying images in a blind setting. How to decompose superimposed images, like rainy images, into distinct source components is a crucial step towards real-world vision systems. We propose a simple yet general Blind Image Decomposition Network (BIDeN) to serve as a strong baseline for future work.
  arXiv  Detail & Related papers  (2021-08-25T17:37:19Z)
• SSH: A Self-Supervised Framework for Image Harmonization [97.16345684998788]
  We propose a novel Self-Supervised Harmonization framework (SSH) that can be trained using just "free" natural images without being edited. Our results show that the proposedSSH outperforms previous state-of-the-art methods in terms of reference metrics, visual quality, and subject user study.
  arXiv  Detail & Related papers  (2021-08-15T19:51:33Z)
• OPA: Object Placement Assessment Dataset [20.791187775546625]
  Image composition aims to generate realistic composite image by inserting an object from one image into another background image. In this paper, we focus on object placement assessment task, which verifies whether a composite image is plausible in terms of the object placement.
  arXiv  Detail & Related papers  (2021-07-05T09:23:53Z)
• Deep Image Compositing [0.0]
  In image editing, the most common task is pasting objects from one image to the other and then adjusting the manifestation of the foreground object with the background object. To achieve this, we are using Generative Adversarial Networks (GANS) GANS is able to decode the color histogram of the foreground and background part of the image and also learns to blend the foreground object with the background.
  arXiv  Detail & Related papers  (2021-03-29T09:23:37Z)
• Bridging Composite and Real: Towards End-to-end Deep Image Matting [88.79857806542006]
  We study the roles of semantics and details for image matting. We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders. Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-30T10:57:13Z)
• Adversarial Image Composition with Auxiliary Illumination [53.89445873577062]
  We propose an Adversarial Image Composition Net (AIC-Net) that achieves realistic image composition. A novel branched generation mechanism is proposed, which disentangles the generation of shadows and the transfer of foreground styles. Experiments on pedestrian and car composition tasks show that the proposed AIC-Net achieves superior composition performance.
  arXiv  Detail & Related papers  (2020-09-17T12:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.