Scaling Painting Style Transfer

• URL: http://arxiv.org/abs/2212.13459v2
• Date: Wed, 26 Jun 2024 13:59:56 GMT
• Title: Scaling Painting Style Transfer
• Authors: Bruno Galerne, Lara Raad, José Lezama, Jean-Michel Morel,
• Abstract summary: Neural style transfer (NST) is a technique that produces an unprecedentedly rich style transfer from a style image to a content image. This paper presents a solution to solve the original global optimization for ultra-high resolution (UHR) images. We show that our method produces style transfer of unmatched quality for such high-resolution painting styles.
• Score: 10.059627473725508
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural style transfer (NST) is a deep learning technique that produces an unprecedentedly rich style transfer from a style image to a content image. It is particularly impressive when it comes to transferring style from a painting to an image. NST was originally achieved by solving an optimization problem to match the global statistics of the style image while preserving the local geometric features of the content image. The two main drawbacks of this original approach is that it is computationally expensive and that the resolution of the output images is limited by high GPU memory requirements. Many solutions have been proposed to both accelerate NST and produce images with larger size. However, our investigation shows that these accelerated methods all compromise the quality of the produced images in the context of painting style transfer. Indeed, transferring the style of a painting is a complex task involving features at different scales, from the color palette and compositional style to the fine brushstrokes and texture of the canvas. This paper provides a solution to solve the original global optimization for ultra-high resolution (UHR) images, enabling multiscale NST at unprecedented image sizes. This is achieved by spatially localizing the computation of each forward and backward passes through the VGG network. Extensive qualitative and quantitative comparisons, as well as a \textcolor{coverletter}{perceptual study}, show that our method produces style transfer of unmatched quality for such high-resolution painting styles. By a careful comparison, we show that state-of-the-art fast methods are still prone to artifacts, thus suggesting that fast painting style transfer remains an open problem. Source code is available at https://github.com/bgalerne/scaling_painting_style_transfer.

Related papers

• Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region [66.75826549444909]
  Stroke-based rendering aims to recreate an image with a set of strokes. We propose Compositional Neural Painter, which predicts the painting region based on the current canvas. We extend our method to stroke-based style transfer with a novel differentiable distance transform loss.
  arXiv  Detail & Related papers  (2023-09-07T06:27:39Z)
• Artistic Arbitrary Style Transfer [1.1279808969568252]
  Arbitrary Style Transfer is a technique used to produce a new image from two images: a content image, and a style image. Balancing the structure and style components has been the major challenge that other state-of-the-art algorithms have tried to solve. In this work, we solved these problems by using a Deep Learning approach using Convolutional Neural Networks.
  arXiv  Detail & Related papers  (2022-12-21T21:34:00Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Interactive Style Transfer: All is Your Palette [74.06681967115594]
  We propose a drawing-like interactive style transfer (IST) method, by which users can interactively create a harmonious-style image. Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image.
  arXiv  Detail & Related papers  (2022-03-25T06:38:46Z)
• Saliency Constrained Arbitrary Image Style Transfer using SIFT and DCNN [22.57205921266602]
  When common neural style transfer methods are used, the textures and colors in the style image are usually transferred imperfectly to the content image. This paper proposes a novel saliency constrained method to reduce or avoid such effects. The experiments show that the saliency maps of source images can help find the correct matching and avoid artifacts.
  arXiv  Detail & Related papers  (2022-01-14T09:00:55Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer [115.13853805292679]
  Artistic style transfer aims at migrating the style from an example image to a content image. Inspired by the common painting process of drawing a draft and revising the details, we introduce a novel feed-forward method named Laplacian Pyramid Network (LapStyle) Our method can synthesize high quality stylized images in real time, where holistic style patterns are properly transferred.
  arXiv  Detail & Related papers  (2021-04-12T11:53:53Z)
• Block Shuffle: A Method for High-resolution Fast Style Transfer with Limited Memory [4.511923587827301]
  Fast Style Transfer is a series of Neural Style Transfer algorithms that use feed-forward neural networks to render input images. Because of the high dimension of the output layer, these networks require much memory for computation. We propose a novel image synthesis method named emphblock shuffle, which converts a single task with high memory consumption to multiple subtasks with low memory consumption.
  arXiv  Detail & Related papers  (2020-08-09T10:33:21Z)
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning [74.09149955786367]
  ArtNet can achieve universal, real-time, and high-quality style transfer on high-resolution images simultaneously. By using ArtNet and S2, our method is 2.3 to 107.4 times faster than state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-06-16T09:50:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.