PHNet: Patch-based Normalization for Portrait Harmonization
- URL: http://arxiv.org/abs/2402.17561v2
- Date: Fri, 8 Mar 2024 09:56:43 GMT
- Title: PHNet: Patch-based Normalization for Portrait Harmonization
- Authors: Karen Efremyan, Elizaveta Petrova, Evgeny Kaskov, and Alexander
Kapitanov
- Abstract summary: A common problem for composite images is the incompatibility of their foreground and background components.
We present a patch-based harmonization network consisting of novel Patch-based normalization blocks and a feature extractor.
Our network achieves state-of-the-art results on the iHarmony4 dataset.
- Score: 41.94295877935867
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: A common problem for composite images is the incompatibility of their
foreground and background components. Image harmonization aims to solve this
problem, making the whole image look more authentic and coherent. Most existing
solutions predict lookup tables (LUTs) or reconstruct images, utilizing various
attributes of composite images. Recent approaches have primarily focused on
employing global transformations like normalization and color curve rendering
to achieve visual consistency, and they often overlook the importance of local
visual coherence. We present a patch-based harmonization network consisting of
novel Patch-based normalization (PN) blocks and a feature extractor based on
statistical color transfer. Extensive experiments demonstrate the network's
high generalization capability for different domains. Our network achieves
state-of-the-art results on the iHarmony4 dataset. Also, we created a new human
portrait harmonization dataset based on FFHQ and checked the proposed method to
show the generalization ability by achieving the best metrics on it. The
benchmark experiments confirm that the suggested patch-based normalization
block and feature extractor effectively improve the network's capability to
harmonize portraits. Our code and model baselines are publicly available.
Related papers
- Breaking the Frame: Image Retrieval by Visual Overlap Prediction [53.17564423756082]
We propose a novel visual place recognition approach, VOP, that efficiently addresses occlusions and complex scenes.
The proposed method enables the identification of visible image sections without requiring expensive feature detection and matching.
arXiv Detail & Related papers (2024-06-23T20:00:20Z) - Distance Weighted Trans Network for Image Completion [52.318730994423106]
We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components.
CNNs are used to augment the local texture information of coarse priors.
DWT blocks are used to recover certain coarse textures and coherent visual structures.
arXiv Detail & Related papers (2023-10-11T12:46:11Z) - Semi-supervised Parametric Real-world Image Harmonization [38.66826236220168]
Learning-based image harmonization techniques are usually trained to undo synthetic random global transformations applied to a masked foreground in a single ground truth photo.
This simulated data does not model many of the important appearance mismatches between foreground and background in real composites.
We propose a new semi-supervised training strategy that addresses this problem and lets us learn complex local appearance harmonization from unpaired real composites.
arXiv Detail & Related papers (2023-03-01T01:09:01Z) - ParaColorizer: Realistic Image Colorization using Parallel Generative
Networks [1.7778609937758327]
Grayscale image colorization is a fascinating application of AI for information restoration.
We present a parallel GAN-based colorization framework.
We show the shortcomings of the non-perceptual evaluation metrics commonly used to assess multi-modal problems.
arXiv Detail & Related papers (2022-08-17T13:49:44Z) - SSH: A Self-Supervised Framework for Image Harmonization [97.16345684998788]
We propose a novel Self-Supervised Harmonization framework (SSH) that can be trained using just "free" natural images without being edited.
Our results show that the proposedSSH outperforms previous state-of-the-art methods in terms of reference metrics, visual quality, and subject user study.
arXiv Detail & Related papers (2021-08-15T19:51:33Z) - A Flexible Framework for Designing Trainable Priors with Adaptive
Smoothing and Game Encoding [57.1077544780653]
We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems.
We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions.
This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
arXiv Detail & Related papers (2020-06-26T08:34:54Z) - Foreground-aware Semantic Representations for Image Harmonization [5.156484100374058]
We propose a novel architecture to utilize the space of high-level features learned by a pre-trained classification network.
We extensively evaluate the proposed method on existing image harmonization benchmark and set up a new state-of-the-art in terms of MSE and PSNR metrics.
arXiv Detail & Related papers (2020-06-01T09:27:20Z) - The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors.
We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS.
To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
arXiv Detail & Related papers (2020-05-16T08:17:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.