AdaCM: Adaptive ColorMLP for Real-Time Universal Photo-realistic Style Transfer

• URL: http://arxiv.org/abs/2212.01567v1
• Date: Sat, 3 Dec 2022 07:56:08 GMT
• Title: AdaCM: Adaptive ColorMLP for Real-Time Universal Photo-realistic Style Transfer
• Authors: Tianwei Lin, Honglin Lin, Fu Li, Dongliang He, Wenhao Wu, Meiling Wang, Xin Li, Yong Liu
• Abstract summary: Photo-realistic style transfer aims at migrating the artistic style from an exemplar style image to a content image, producing a result image without spatial distortions or unrealistic artifacts. We propose the textbfAdaptive ColorMLP (AdaCM), an effective and efficient framework for universal photo-realistic style transfer.
• Score: 53.41350013698697
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photo-realistic style transfer aims at migrating the artistic style from an exemplar style image to a content image, producing a result image without spatial distortions or unrealistic artifacts. Impressive results have been achieved by recent deep models. However, deep neural network based methods are too expensive to run in real-time. Meanwhile, bilateral grid based methods are much faster but still contain artifacts like overexposure. In this work, we propose the \textbf{Adaptive ColorMLP (AdaCM)}, an effective and efficient framework for universal photo-realistic style transfer. First, we find the complex non-linear color mapping between input and target domain can be efficiently modeled by a small multi-layer perceptron (ColorMLP) model. Then, in \textbf{AdaCM}, we adopt a CNN encoder to adaptively predict all parameters for the ColorMLP conditioned on each input content and style image pair. Experimental results demonstrate that AdaCM can generate vivid and high-quality stylization results. Meanwhile, our AdaCM is ultrafast and can process a 4K resolution image in 6ms on one V100 GPU.

Related papers

• PixelShuffler: A Simple Image Translation Through Pixel Rearrangement [0.0]
  Style transfer is a widely researched application of image-to-image translation, where the goal is to synthesize an image that combines the content of one image with the style of another. Existing state-of-the-art methods often rely on complex neural networks, including diffusion models and language models, to achieve high-quality style transfer. We propose a novel pixel shuffle method that addresses the image-to-image translation problem generally with a specific demonstrative application in style transfer.
  arXiv  Detail & Related papers  (2024-10-03T22:08:41Z)
• Improving Human-Object Interaction Detection via Virtual Image Learning [68.56682347374422]
  Human-Object Interaction (HOI) detection aims to understand the interactions between humans and objects. In this paper, we propose to alleviate the impact of such an unbalanced distribution via Virtual Image Leaning (VIL) A novel label-to-image approach, Multiple Steps Image Creation (MUSIC), is proposed to create a high-quality dataset that has a consistent distribution with real images.
  arXiv  Detail & Related papers  (2023-08-04T10:28:48Z)
• HQ3DAvatar: High Quality Controllable 3D Head Avatar [65.70885416855782]
  This paper presents a novel approach to building highly photorealistic digital head avatars. Our method learns a canonical space via an implicit function parameterized by a neural network. At test time, our method is driven by a monocular RGB video.
  arXiv  Detail & Related papers  (2023-03-25T13:56:33Z)
• Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping [17.692674513446153]
  We show that information about the style can be distilled from collections of image pairs and encoded into a 2- or 3-dimensional vector. We show that such a network is more effective than PCA or VAE at normalizing image style in low-dimensional space.
  arXiv  Detail & Related papers  (2022-09-30T01:26:16Z)
• UV Volumes for Real-time Rendering of Editable Free-view Human Performance [35.089358945669865]
  UV Volumes is an approach that can render an editable free-view video of a human performer in real-time. It is achieved by removing the high-frequency (i.e., non-smooth) human textures from the 3D volume and encoding them into a 2D neural texture stack. Experiments on CMU Panoptic, ZJU Mocap, and H36M datasets show that our model can render 900 * 500 images in 40 fps.
  arXiv  Detail & Related papers  (2022-03-27T21:54:36Z)
• Interactive Style Transfer: All is Your Palette [74.06681967115594]
  We propose a drawing-like interactive style transfer (IST) method, by which users can interactively create a harmonious-style image. Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image.
  arXiv  Detail & Related papers  (2022-03-25T06:38:46Z)
• FlexIT: Towards Flexible Semantic Image Translation [59.09398209706869]
  We propose FlexIT, a novel method which can take any input image and a user-defined text instruction for editing. First, FlexIT combines the input image and text into a single target point in the CLIP multimodal embedding space. We iteratively transform the input image toward the target point, ensuring coherence and quality with a variety of novel regularization terms.
  arXiv  Detail & Related papers  (2022-03-09T13:34:38Z)
• UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation [0.0]
  We propose a photorealistic style transfer network to emphasize the natural effect of photorealistic image stylization. In particular, an encoder based on the dense block and a decoder form a symmetrical structure of U-Net are jointly staked to realize an effective feature extraction and image reconstruction.
  arXiv  Detail & Related papers  (2021-08-13T08:06:29Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer [18.455002563426262]
  Photorealistic style transfer is the task of transferring the artistic style of an image onto a content target, producing a result that is plausibly taken with a camera. Recent approaches, based on deep neural networks, produce impressive results but are either too slow to run at practical resolutions, or still contain objectionable artifacts. We propose a new end-to-end model for photorealistic style transfer that is both fast and inherently generates photorealistic results.
  arXiv  Detail & Related papers  (2020-04-23T03:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.