Deep Image Compositing

• URL: http://arxiv.org/abs/2011.02146v1
• Date: Wed, 4 Nov 2020 06:12:24 GMT
• Title: Deep Image Compositing
• Authors: He Zhang, Jianming Zhang, Federico Perazzi, Zhe Lin, Vishal M. Patel
• Abstract summary: We propose a new method which can automatically generate high-quality image composites without any user input. Inspired by Laplacian pyramid blending, a dense-connected multi-stream fusion network is proposed to effectively fuse the information from the foreground and background images. Experiments show that the proposed method can automatically generate high-quality composites and outperforms existing methods both qualitatively and quantitatively.
• Score: 93.75358242750752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image compositing is a task of combining regions from different images to compose a new image. A common use case is background replacement of portrait images. To obtain high quality composites, professionals typically manually perform multiple editing steps such as segmentation, matting and foreground color decontamination, which is very time consuming even with sophisticated photo editing tools. In this paper, we propose a new method which can automatically generate high-quality image compositing without any user input. Our method can be trained end-to-end to optimize exploitation of contextual and color information of both foreground and background images, where the compositing quality is considered in the optimization. Specifically, inspired by Laplacian pyramid blending, a dense-connected multi-stream fusion network is proposed to effectively fuse the information from the foreground and background images at different scales. In addition, we introduce a self-taught strategy to progressively train from easy to complex cases to mitigate the lack of training data. Experiments show that the proposed method can automatically generate high-quality composites and outperforms existing methods both qualitatively and quantitatively.

Related papers

• Fusion from Decomposition: A Self-Supervised Approach for Image Fusion and Beyond [74.96466744512992]
  The essence of image fusion is to integrate complementary information from source images. DeFusion++ produces versatile fused representations that can enhance the quality of image fusion and the effectiveness of downstream high-level vision tasks.
  arXiv  Detail & Related papers  (2024-10-16T06:28:49Z)
• FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior [50.0535198082903]
  We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. We showcase the potential of utilizing the powerful generative prior inherent in large-scale pre-trained diffusion models to accomplish generic image composition.
  arXiv  Detail & Related papers  (2024-07-06T03:35:43Z)
• Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
  We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• Text2Layer: Layered Image Generation using Latent Diffusion Model [12.902259486204898]
  We propose to generate layered images from a layered image generation perspective. To achieve layered image generation, we train an autoencoder that is able to reconstruct layered images. Experimental results show that the proposed method is able to generate high-quality layered images.
  arXiv  Detail & Related papers  (2023-07-19T06:56:07Z)
• Deep Image Matting: A Comprehensive Survey [85.77905619102802]
  This paper presents a review of recent advancements in image matting in the era of deep learning. We focus on two fundamental sub-tasks: auxiliary input-based image matting and automatic image matting. We discuss relevant applications of image matting and highlight existing challenges and potential opportunities for future research.
  arXiv  Detail & Related papers  (2023-04-10T15:48:55Z)
• TransFuse: A Unified Transformer-based Image Fusion Framework using Self-supervised Learning [5.849513679510834]
  Image fusion is a technique to integrate information from multiple source images with complementary information to improve the richness of a single image. Two-stage methods avoid the need of large amount of task-specific training data by training encoder-decoder network on large natural image datasets. We propose a destruction-reconstruction based self-supervised training scheme to encourage the network to learn task-specific features.
  arXiv  Detail & Related papers  (2022-01-19T07:30:44Z)
• Semantic Image Fusion [2.4366811507669124]
  This paper proposes a novel system for the semantic combination of visual content using pre-trained CNN network architectures. Simple "choose maximum" and "local majority" filter based fusion rules are utilised for feature map fusion. The developed methods are able to give equivalent low-level fusion performance to state of the art methods.
  arXiv  Detail & Related papers  (2021-10-13T13:15:16Z)
• Bridging Composite and Real: Towards End-to-end Deep Image Matting [88.79857806542006]
  We study the roles of semantics and details for image matting. We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders. Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-30T10:57:13Z)
• AlphaNet: An Attention Guided Deep Network for Automatic Image Matting [0.0]
  We propose an end to end solution for image matting i.e. high-precision extraction of foreground objects from natural images. We propose a method that assimilates semantic segmentation and deep image matting processes into a single network to generate semantic mattes. We also construct a fashion e-commerce focused dataset with high-quality alpha mattes to facilitate the training and evaluation for image matting.
  arXiv  Detail & Related papers  (2020-03-07T17:25:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.