Dense Pixel-to-Pixel Harmonization via Continuous Image Representation
- URL: http://arxiv.org/abs/2303.01681v2
- Date: Thu, 30 Nov 2023 13:08:34 GMT
- Title: Dense Pixel-to-Pixel Harmonization via Continuous Image Representation
- Authors: Jianqi Chen, Yilan Zhang, Zhengxia Zou, Keyan Chen, Zhenwei Shi
- Abstract summary: We propose a novel image Harmonization method based on Implicit neural Networks (HINet)
Inspired by the Retinex theory, we decouple the harmonizations into two parts to respectively capture the content and environment of composite images.
Extensive experiments have demonstrated the effectiveness of our method compared with state-of-the-art methods.
- Score: 22.984119094424056
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: High-resolution (HR) image harmonization is of great significance in
real-world applications such as image synthesis and image editing. However, due
to the high memory costs, existing dense pixel-to-pixel harmonization methods
are mainly focusing on processing low-resolution (LR) images. Some recent works
resort to combining with color-to-color transformations but are either limited
to certain resolutions or heavily depend on hand-crafted image filters. In this
work, we explore leveraging the implicit neural representation (INR) and
propose a novel image Harmonization method based on Implicit neural Networks
(HINet), which to the best of our knowledge, is the first dense pixel-to-pixel
method applicable to HR images without any hand-crafted filter design. Inspired
by the Retinex theory, we decouple the MLPs into two parts to respectively
capture the content and environment of composite images. A Low-Resolution Image
Prior (LRIP) network is designed to alleviate the Boundary Inconsistency
problem, and we also propose new designs for the training and inference
process. Extensive experiments have demonstrated the effectiveness of our
method compared with state-of-the-art methods. Furthermore, some interesting
and practical applications of the proposed method are explored. Our code is
available at https://github.com/WindVChen/INR-Harmonization.
Related papers
- Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
Digital image forensics plays a crucial role in image authentication and manipulation localization.
This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts.
Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
arXiv Detail & Related papers (2023-09-30T02:54:51Z) - DCCF: Deep Comprehensible Color Filter Learning Framework for
High-Resolution Image Harmonization [14.062386668676533]
We propose a novel Deep Comprehensible Color Filter (DCCF) learning framework for high-resolution image harmonization.
DCCF learns four human comprehensible neural filters (i.e. hue, saturation, value and attentive rendering filters) in an end-to-end manner.
It outperforms state-of-the-art post-processing method on iHarmony4 dataset on images' full-resolutions by achieving 7.63% and 1.69% relative improvements on MSE and PSNR respectively.
arXiv Detail & Related papers (2022-07-11T11:42:10Z) - Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image
Denoising [50.039949798156826]
This paper tackles the challenging problem of hyperspectral (HS) image denoising.
We propose rank-enhanced low-dimensional convolution set (Re-ConvSet)
We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
arXiv Detail & Related papers (2022-07-09T13:35:12Z) - High-Resolution Image Harmonization via Collaborative Dual
Transformations [13.9962809174055]
We propose a high-resolution image harmonization network with Collaborative Dual Transformation (CDTNet)
Our CDTNet consists of a low-resolution generator for pixel-to-pixel transformation, a color mapping module for RGB-to-RGB transformation, and a refinement module to take advantage of both.
arXiv Detail & Related papers (2021-09-14T13:18:58Z) - Spatial-Separated Curve Rendering Network for Efficient and
High-Resolution Image Harmonization [59.19214040221055]
We propose a novel spatial-separated curve rendering network (S$2$CRNet) for efficient and high-resolution image harmonization.
The proposed method reduces more than 90% parameters compared with previous methods.
Our method can work smoothly on higher resolution images in real-time which is more than 10$times$ faster than the existing methods.
arXiv Detail & Related papers (2021-09-13T07:20:16Z) - LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single
Image Super-Resolution and Beyond [75.37541439447314]
Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version.
This paper proposes a linearly-assembled pixel-adaptive regression network (LAPAR) to strike a sweet spot of deep model complexity and resulting SISR quality.
arXiv Detail & Related papers (2021-05-21T15:47:18Z) - Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline
for a Pixel-bin Image Sensor [0.883717274344425]
Pixel binning is considered one of the most prominent solutions to tackle the hardware limitation of smartphone cameras.
In this paper, we tackle the challenges of joint demosaicing and denoising (JDD) on such an image sensor by introducing a novel learning-based method.
The proposed method is guided by a multi-term objective function, including two novel perceptual losses to produce visually plausible images.
arXiv Detail & Related papers (2021-04-19T15:41:28Z) - Adversarial Generation of Continuous Images [31.92891885615843]
In this paper, we propose two novel architectural techniques for building INR-based image decoders.
We use them to build a state-of-the-art continuous image GAN.
Our proposed INR-GAN architecture improves the performance of continuous image generators by several times.
arXiv Detail & Related papers (2020-11-24T11:06:40Z) - The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors.
We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS.
To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
arXiv Detail & Related papers (2020-05-16T08:17:44Z) - Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task.
We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network.
Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
arXiv Detail & Related papers (2020-03-15T11:04:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.