Painterly Image Harmonization via Adversarial Residual Learning
- URL: http://arxiv.org/abs/2311.08646v1
- Date: Wed, 15 Nov 2023 01:53:46 GMT
- Title: Painterly Image Harmonization via Adversarial Residual Learning
- Authors: Xudong Wang, Li Niu, Junyan Cao, Yan Hong, Liqing Zhang
- Abstract summary: painterly image aims to transfer the style of background painting to the foreground object.
In this work, we employ adversarial learning to bridge the domain gap between foreground feature map and background feature map.
- Score: 37.78751164466694
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image compositing plays a vital role in photo editing. After inserting a
foreground object into another background image, the composite image may look
unnatural and inharmonious. When the foreground is photorealistic and the
background is an artistic painting, painterly image harmonization aims to
transfer the style of background painting to the foreground object, which is a
challenging task due to the large domain gap between foreground and background.
In this work, we employ adversarial learning to bridge the domain gap between
foreground feature map and background feature map. Specifically, we design a
dual-encoder generator, in which the residual encoder produces the residual
features added to the foreground feature map from main encoder. Then, a
pixel-wise discriminator plays against the generator, encouraging the refined
foreground feature map to be indistinguishable from background feature map.
Extensive experiments demonstrate that our method could achieve more harmonious
and visually appealing results than previous methods.
Related papers
- DiffUHaul: A Training-Free Method for Object Dragging in Images [78.93531472479202]
We propose a training-free method, dubbed DiffUHaul, for the object dragging task.
We first apply attention masking in each denoising step to make the generation more disentangled across different objects.
In the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance.
arXiv Detail & Related papers (2024-06-03T17:59:53Z) - Painterly Image Harmonization by Learning from Painterly Objects [35.23590833646526]
We learn a mapping from background style and object information to object style based on painterly objects in artistic paintings.
With the learnt mapping, we can hallucinate the target style of composite object, which is used to harmonize encoder feature maps to produce the harmonized image.
arXiv Detail & Related papers (2023-12-15T23:36:44Z) - Cylin-Painting: Seamless {360\textdegree} Panoramic Image Outpainting
and Beyond [136.18504104345453]
We present a Cylin-Painting framework that involves meaningful collaborations between inpainting and outpainting.
The proposed algorithm can be effectively extended to other panoramic vision tasks, such as object detection, depth estimation, and image super-resolution.
arXiv Detail & Related papers (2022-04-18T21:18:49Z) - Contextual Attention Mechanism, SRGAN Based Inpainting System for
Eliminating Interruptions from Images [2.894944733573589]
We propose an end-to-end pipeline for inpainting images using a complete Machine Learning approach.
We first use the YOLO model to automatically identify and localize the object we wish to remove from the image.
After this, we provide the masked image and original image to the GAN model which uses the Contextual Attention method to fill in the region.
arXiv Detail & Related papers (2022-04-06T05:51:04Z) - Location-Free Camouflage Generation Network [82.74353843283407]
Camouflage is a common visual phenomenon, which refers to hiding the foreground objects into the background images, making them briefly invisible to the human eye.
This paper proposes a novel Location-free Camouflage Generation Network (LCG-Net) that fuse high-level features of foreground and background image, and generate result by one inference.
Experiments show that our method has results as satisfactory as state-of-the-art in the single-appearance regions and are less likely to be completely invisible, but far exceed the quality of the state-of-the-art in the multi-appearance regions.
arXiv Detail & Related papers (2022-03-18T10:33:40Z) - Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with
Conditional StyleGAN [88.62422914645066]
We present an algorithm for re-rendering a person from a single image under arbitrary poses.
Existing methods often have difficulties in hallucinating occluded contents photo-realistically while preserving the identity and fine details in the source image.
We show that our method compares favorably against the state-of-the-art algorithms in both quantitative evaluation and visual comparison.
arXiv Detail & Related papers (2021-09-13T17:59:33Z) - BargainNet: Background-Guided Domain Translation for Image Harmonization [26.370523451625466]
Unharmonious foreground and background downgrade the quality of composite image.
Image harmonization, which adjusts the foreground to improve the consistency, is an essential yet challenging task.
We propose an image harmonization network with a novel domain code extractor and well-tailored triplet losses.
arXiv Detail & Related papers (2020-09-19T05:14:08Z) - BachGAN: High-Resolution Image Synthesis from Salient Object Layout [78.51640906030244]
We propose a new task towards more practical application for image generation - high-quality image synthesis from salient object layout.
Two main challenges spring from this new task: (i) how to generate fine-grained details and realistic textures without segmentation map input; and (ii) how to create a background and weave it seamlessly into standalone objects.
By generating the hallucinated background representation dynamically, our model can synthesize high-resolution images with both photo-realistic foreground and integral background.
arXiv Detail & Related papers (2020-03-26T00:54:44Z) - Unsupervised Disentanglement of Pose, Appearance and Background from
Images and Videos [44.93648211794362]
Unsupervised landmark learning is the task of learning semantic keypoint-like representations without the use of expensive input keypoint-level annotations.
A popular approach is to factorize an image into a pose and appearance data stream, then to reconstruct the image from the factorized components.
This work explores the effects of factorizing the reconstruction task into separate foreground and background reconstructions.
arXiv Detail & Related papers (2020-01-26T20:59:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.