Multi-Curve Translator for Real-Time High-Resolution Image-to-Image Translation

• URL: http://arxiv.org/abs/2203.07756v1
• Date: Tue, 15 Mar 2022 10:06:39 GMT
• Title: Multi-Curve Translator for Real-Time High-Resolution Image-to-Image Translation
• Authors: Yuda Song, Hui Qian, Xin Du
• Abstract summary: Multi-Curve Translator (MCT) predicts translated pixels for corresponding input pixels and neighboring pixels. MCT makes it possible to feed the network only the downsampled image to perform the mapping for the full-resolution image. MCT variants can process 4K images in real-time and achieve comparable or even better performance than the base models.
• Score: 24.651984136294242
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The dominant image-to-image translation methods are based on fully convolutional networks, which extract and translate an image's features and then reconstruct the image. However, they have unacceptable computational costs when working with high-resolution images. To this end, we present the Multi-Curve Translator (MCT), which not only predicts the translated pixels for the corresponding input pixels but also for their neighboring pixels. And if a high-resolution image is downsampled to its low-resolution version, the lost pixels are the remaining pixels' neighboring pixels. So MCT makes it possible to feed the network only the downsampled image to perform the mapping for the full-resolution image, which can dramatically lower the computational cost. Besides, MCT is a plug-in approach that utilizes existing base models and requires only replacing their output layers. Experiments demonstrate that the MCT variants can process 4K images in real-time and achieve comparable or even better performance than the base models on various image-to-image translation tasks.

Related papers

• Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization [4.349838917565205]
  We introduce a Dense Normalization layer designed to estimate pixel-level statistical moments. This approach effectively diminishes tiling artifacts while concurrently preserving local color and hue contrasts. Our work paves the way for future exploration in handling images of arbitrary resolutions within the realm of unpaired image-to-image translation.
  arXiv  Detail & Related papers  (2024-07-05T04:14:50Z)
• Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization [4.8454936010479335]
  We propose a Multi-view Pixel-wise Contrastive algorithm (MPC) for image forgery localization. Specifically, we first pre-train the backbone network with the supervised contrastive loss. Then the localization head is fine-tuned using the cross-entropy loss, resulting in a better pixel localizer.
  arXiv  Detail & Related papers  (2024-06-19T13:51:52Z)
• High-Resolution Image Translation Model Based on Grayscale Redefinition [3.6996084306161277]
  We propose an innovative method for image translation between different domains. For high-resolution image translation tasks, we use a grayscale adjustment method to achieve pixel-level translation. For other tasks, we utilize the Pix2PixHD model with a coarse-to-fine generator, multi-scale discriminator, and improved loss to enhance the image translation performance.
  arXiv  Detail & Related papers  (2024-03-26T12:21:47Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [63.54342601757723]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution [91.56337748920662]
  Diffusion probabilistic models (DPM) have been widely adopted in image-to-image translation. We propose a simple but non-trivial DPM-based super-resolution post-process framework,i.e., cDPMSR. Our method surpasses prior attempts on both qualitative and quantitative results.
  arXiv  Detail & Related papers  (2023-02-14T15:13:33Z)
• PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation [84.97160975101718]
  We propose a novel generative adversarial network, PI-Trans, which consists of a novel Parallel-ConvMLP module and an Implicit Transformation module at multiple semantic levels. PI-Trans achieves the best qualitative and quantitative performance by a large margin compared to the state-of-the-art methods on two challenging datasets.
  arXiv  Detail & Related papers  (2022-07-09T10:35:44Z)
• Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
  convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data. Transformers have shown significant performance gains on natural language and high-level vision tasks. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:10Z)
• LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond [75.37541439447314]
  Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. This paper proposes a linearly-assembled pixel-adaptive regression network (LAPAR) to strike a sweet spot of deep model complexity and resulting SISR quality.
  arXiv  Detail & Related papers  (2021-05-21T15:47:18Z)
• High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network [23.981019687483506]
  We focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. We propose a Laplacian Pyramid Translation Network (N) to simultaneously perform these two tasks. Our model avoids most of the heavy computation consumed by processing high-resolution feature maps and faithfully preserves the image details.
  arXiv  Detail & Related papers  (2021-05-19T15:05:22Z)
• Spatially-Adaptive Pixelwise Networks for Fast Image Translation [57.359250882770525]
  We introduce a new generator architecture, aimed at fast and efficient high-resolution image-to-image translation. We use pixel-wise networks; that is, each pixel is processed independently of others. Our model is up to 18x faster than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-12-05T10:02:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.