High-Resolution Image Harmonization via Collaborative Dual Transformations

• URL: http://arxiv.org/abs/2109.06671v1
• Date: Tue, 14 Sep 2021 13:18:58 GMT
• Title: High-Resolution Image Harmonization via Collaborative Dual Transformations
• Authors: Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, Liqing Zhang
• Abstract summary: We propose a high-resolution image harmonization network with Collaborative Dual Transformation (CDTNet) Our CDTNet consists of a low-resolution generator for pixel-to-pixel transformation, a color mapping module for RGB-to-RGB transformation, and a refinement module to take advantage of both.
• Score: 13.9962809174055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given a composite image, image harmonization aims to adjust the foreground to make it compatible with the background. High-resolution image harmonization is in high demand, but still remains unexplored. Conventional image harmonization methods learn global RGB-to-RGB transformation which could effortlessly scale to high resolution, but ignore diverse local context. Recent deep learning methods learn the dense pixel-to-pixel transformation which could generate harmonious outputs, but are highly constrained in low resolution. In this work, we propose a high-resolution image harmonization network with Collaborative Dual Transformation (CDTNet) to combine pixel-to-pixel transformation and RGB-to-RGB transformation coherently in an end-to-end framework. Our CDTNet consists of a low-resolution generator for pixel-to-pixel transformation, a color mapping module for RGB-to-RGB transformation, and a refinement module to take advantage of both. Extensive experiments on high-resolution image harmonization dataset demonstrate that our CDTNet strikes a good balance between efficiency and effectiveness.

Related papers

• Beyond Learned Metadata-based Raw Image Reconstruction [86.1667769209103]
  Raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels. They are not widely adopted by general users due to their substantial storage requirements. We propose a novel framework that learns a compact representation in the latent space, serving as metadata.
  arXiv  Detail & Related papers  (2023-06-21T06:59:07Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• Dense Pixel-to-Pixel Harmonization via Continuous Image Representation [22.984119094424056]
  We propose a novel image Harmonization method based on Implicit neural Networks (HINet) Inspired by the Retinex theory, we decouple the harmonizations into two parts to respectively capture the content and environment of composite images. Extensive experiments have demonstrated the effectiveness of our method compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-03T02:52:28Z)
• Raw Image Reconstruction with Learned Compact Metadata [61.62454853089346]
  We propose a novel framework to learn a compact representation in the latent space serving as the metadata in an end-to-end manner. We show how the proposed raw image compression scheme can adaptively allocate more bits to image regions that are important from a global perspective.
  arXiv  Detail & Related papers  (2023-02-25T05:29:45Z)
• Hyperspectral Image Super Resolution with Real Unaligned RGB Guidance [11.711656319221072]
  We propose an HSI fusion network with heterogenous feature extractions, multi-stage feature alignments, and attentive feature fusion. Our method obtains a clear improvement over existing single-image and fusion-based super-resolution methods on quantitative assessment as well as visual comparison.
  arXiv  Detail & Related papers  (2023-02-13T11:56:45Z)
• Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection [16.64781797503128]
  RGB-thermal object detection (RGB-T SOD) aims to locate the common prominent objects of an aligned visible and thermal infrared image pair. In this paper, we propose a novel mirror complementary Transformer network (MCNet) for RGB-T SOD. Experiments on benchmark and VT723 datasets show that the proposed method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-07-07T20:26:09Z)
• FRIH: Fine-grained Region-aware Image Harmonization [49.420765789360836]
  We propose a novel global-local two stages framework for Fine-grained Region-aware Image Harmonization (FRIH) Our algorithm achieves the best performance on iHarmony4 dataset (PSNR is 38.19 dB) with a lightweight model.
  arXiv  Detail & Related papers  (2022-05-13T04:50:26Z)
• Transformer-based Network for RGB-D Saliency Detection [82.6665619584628]
  Key to RGB-D saliency detection is to fully mine and fuse information at multiple scales across the two modalities. We show that transformer is a uniform operation which presents great efficacy in both feature fusion and feature enhancement. Our proposed network performs favorably against state-of-the-art RGB-D saliency detection methods.
  arXiv  Detail & Related papers  (2021-12-01T15:53:58Z)
• Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization [59.19214040221055]
  We propose a novel spatial-separated curve rendering network (S$2$CRNet) for efficient and high-resolution image harmonization. The proposed method reduces more than 90% parameters compared with previous methods. Our method can work smoothly on higher resolution images in real-time which is more than 10$times$ faster than the existing methods.
  arXiv  Detail & Related papers  (2021-09-13T07:20:16Z)
• Discrete Cosine Transform Network for Guided Depth Map Super-Resolution [19.86463937632802]
  The goal is to use high-resolution (HR) RGB images to provide extra information on edges and object contours, so that low-resolution depth maps can be upsampled to HR ones. We propose an advanced Discrete Cosine Transform Network (DCTNet), which is composed of four components. We show that our method can generate accurate and HR depth maps, surpassing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-04-14T17:01:03Z)
• Fast Generation of High Fidelity RGB-D Images by Deep-Learning with Adaptive Convolution [10.085742605397124]
  We propose a deep-learning based approach to efficiently generate RGB-D images with completed information in high resolution. As an end-to-end approach, high fidelity RGB-D images can be generated efficiently at the rate of around 21 frames per second.
  arXiv  Detail & Related papers  (2020-02-12T16:14:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.