High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network

• URL: http://arxiv.org/abs/2105.09188v1
• Date: Wed, 19 May 2021 15:05:22 GMT
• Title: High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network
• Authors: Jie Liang, Hui Zeng, Lei Zhang
• Abstract summary: We focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. We propose a Laplacian Pyramid Translation Network (N) to simultaneously perform these two tasks. Our model avoids most of the heavy computation consumed by processing high-resolution feature maps and faithfully preserves the image details.
• Score: 23.981019687483506
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Existing image-to-image translation (I2IT) methods are either constrained to low-resolution images or long inference time due to their heavy computational burden on the convolution of high-resolution feature maps. In this paper, we focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. Specifically, we reveal that the attribute transformations, such as illumination and color manipulation, relate more to the low-frequency component, while the content details can be adaptively refined on high-frequency components. We consequently propose a Laplacian Pyramid Translation Network (LPTN) to simultaneously perform these two tasks, where we design a lightweight network for translating the low-frequency component with reduced resolution and a progressive masking strategy to efficiently refine the high-frequency ones. Our model avoids most of the heavy computation consumed by processing high-resolution feature maps and faithfully preserves the image details. Extensive experimental results on various tasks demonstrate that the proposed method can translate 4K images in real-time using one normal GPU while achieving comparable transformation performance against existing methods. Datasets and codes are available: https://github.com/csjliang/LPTN.

Related papers

• 4K-Resolution Photo Exposure Correction at 125 FPS with ~8K Parameters [9.410502389242815]
  In this paper, we propose extremely light-weight (with only 8K parameters) Multi-Scale Linear Transformation (MSLT) networks. MSLT networks can process 4K-resolution sRGB images at 125 Frame-Per-Second (FPS) by a Titan GTX GPU. Experiments on two benchmark datasets demonstrate the efficiency of our MSLTs against the state-of-the-arts on photo exposure correction.
  arXiv  Detail & Related papers  (2023-11-15T08:01:12Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• T-former: An Efficient Transformer for Image Inpainting [50.43302925662507]
  A class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields. In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion, and based on this attention, a network called $T$-former is designed for image inpainting. Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity.
  arXiv  Detail & Related papers  (2023-05-12T04:10:42Z)
• CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying [52.91778151771145]
  In this paper, we try to break the limitations for the first time thanks to the recent development of continuous implicit representation. Experiments show that the proposed method achieves real-time performance on the 2048$times$2048 images using a single GTX 2080 Ti GPU.
  arXiv  Detail & Related papers  (2023-03-15T11:13:51Z)
• Multi-Curve Translator for Real-Time High-Resolution Image-to-Image Translation [24.651984136294242]
  Multi-Curve Translator (MCT) predicts translated pixels for corresponding input pixels and neighboring pixels. MCT makes it possible to feed the network only the downsampled image to perform the mapping for the full-resolution image. MCT variants can process 4K images in real-time and achieve comparable or even better performance than the base models.
  arXiv  Detail & Related papers  (2022-03-15T10:06:39Z)
• SDWNet: A Straight Dilated Network with Wavelet Transformation for Image Deblurring [23.86692375792203]
  Image deblurring is a computer vision problem that aims to recover a sharp image from a blurred image. Our model uses dilated convolution to enable the obtainment of the large receptive field with high spatial resolution. We propose a novel module using the wavelet transform, which effectively helps the network to recover clear high-frequency texture details.
  arXiv  Detail & Related papers  (2021-10-12T07:58:10Z)
• Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization [59.19214040221055]
  We propose a novel spatial-separated curve rendering network (S$2$CRNet) for efficient and high-resolution image harmonization. The proposed method reduces more than 90% parameters compared with previous methods. Our method can work smoothly on higher resolution images in real-time which is more than 10$times$ faster than the existing methods.
  arXiv  Detail & Related papers  (2021-09-13T07:20:16Z)
• Fast Monte Carlo Rendering via Multi-Resolution Sampling [6.203886925467029]
  We present a hybrid rendering method to speed up Monte Carlo rendering algorithms. Our method generates two versions of a rendering: one at a low resolution with a high sample rate (LRHS) and the other at a high resolution with a low sample rate (HRLS)
  arXiv  Detail & Related papers  (2021-06-24T07:35:27Z)
• Spatially-Adaptive Pixelwise Networks for Fast Image Translation [57.359250882770525]
  We introduce a new generator architecture, aimed at fast and efficient high-resolution image-to-image translation. We use pixel-wise networks; that is, each pixel is processed independently of others. Our model is up to 18x faster than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-12-05T10:02:03Z)
• Efficient texture-aware multi-GAN for image inpainting [5.33024001730262]
  Recent GAN-based (Generative adversarial networks) inpainting methods show remarkable improvements. We propose a multi-GAN architecture improving both the performance and rendering efficiency.
  arXiv  Detail & Related papers  (2020-09-30T14:58:03Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.