NeAT: Neural Artistic Tracing for Beautiful Style Transfer

• URL: http://arxiv.org/abs/2304.05139v1
• Date: Tue, 11 Apr 2023 11:08:13 GMT
• Title: NeAT: Neural Artistic Tracing for Beautiful Style Transfer
• Authors: Dan Ruta, Andrew Gilbert, John Collomosse, Eli Shechtman, Nicholas Kolkin
• Abstract summary: Style transfer is the task of reproducing semantic contents of a source image in the artistic style of a second target image. We present NeAT, a new state-of-the art feed-forward style transfer method. We use BBST-4M to improve and measure the generalization of NeAT across a huge variety of styles.
• Score: 29.38791171225834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Style transfer is the task of reproducing the semantic contents of a source image in the artistic style of a second target image. In this paper, we present NeAT, a new state-of-the art feed-forward style transfer method. We re-formulate feed-forward style transfer as image editing, rather than image generation, resulting in a model which improves over the state-of-the-art in both preserving the source content and matching the target style. An important component of our model's success is identifying and fixing "style halos", a commonly occurring artefact across many style transfer techniques. In addition to training and testing on standard datasets, we introduce the BBST-4M dataset, a new, large scale, high resolution dataset of 4M images. As a component of curating this data, we present a novel model able to classify if an image is stylistic. We use BBST-4M to improve and measure the generalization of NeAT across a huge variety of styles. Not only does NeAT offer state-of-the-art quality and generalization, it is designed and trained for fast inference at high resolution.

Related papers

• MuseumMaker: Continual Style Customization without Catastrophic Forgetting [50.12727620780213]
  We propose MuseumMaker, a method that enables the synthesis of images by following a set of customized styles in a never-end manner. When facing with a new customization style, we develop a style distillation loss module to extract and learn the styles of the training data for new image generation. It can minimize the learning biases caused by content of new training images, and address the catastrophic overfitting issue induced by few-shot images.
  arXiv  Detail & Related papers  (2024-04-25T13:51:38Z)
• Towards Highly Realistic Artistic Style Transfer via Stable Diffusion with Step-aware and Layer-aware Prompt [12.27693060663517]
  Artistic style transfer aims to transfer the learned artistic style onto an arbitrary content image, generating artistic stylized images. We propose a novel pre-trained diffusion-based artistic style transfer method, called LSAST. Our proposed method can generate more highly realistic artistic stylized images than the state-of-the-art artistic style transfer methods.
  arXiv  Detail & Related papers  (2024-04-17T15:28:53Z)
• ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank [9.99530386586636]
  Artistic style transfer aims to repaint the content image with the learned artistic style. Existing artistic style transfer methods can be divided into two categories: small model-based approaches and pre-trained large-scale model-based approaches. We propose ArtBank, a novel artistic style transfer framework, to generate highly realistic stylized images.
  arXiv  Detail & Related papers  (2023-12-11T05:53:40Z)
• TSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style Transfer [22.16475032434281]
  Artistic style transfer aims to create new artistic images by rendering a given photograph with the target artistic style. Existing methods learn styles simply based on global statistics or local patches, lacking careful consideration of the drawing process in practice. We propose a Two-Stage Statistics-Aware Transformation (TSSAT) module, which first builds the global style foundation by aligning the global statistics of content and style features. To further enhance both content and style representations, we introduce two novel losses: an attention-based content loss and a patch-based style loss.
  arXiv  Detail & Related papers  (2023-09-12T07:02:13Z)
• DIFF-NST: Diffusion Interleaving For deFormable Neural Style Transfer [27.39248034592382]
  We propose using a new class of models to perform style transfer while enabling deformable style transfer. We show how leveraging the priors of these models can expose new artistic controls at inference time.
  arXiv  Detail & Related papers  (2023-07-09T12:13:43Z)
• StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model [64.26721402514957]
  We propose StylerDALLE, a style transfer method that uses natural language to describe abstract art styles. Specifically, we formulate the language-guided style transfer task as a non-autoregressive token sequence translation. To incorporate style information, we propose a Reinforcement Learning strategy with CLIP-based language supervision.
  arXiv  Detail & Related papers  (2023-03-16T12:44:44Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning [89.86971464234533]
  Cross-Domain Few-Shot Learning (CD-FSL) is a recently emerging task that tackles few-shot learning across different domains. We propose a novel model-agnostic meta Style Adversarial training (StyleAdv) method together with a novel style adversarial attack method. Our method is gradually robust to the visual styles, thus boosting the generalization ability for novel target datasets.
  arXiv  Detail & Related papers  (2023-02-18T11:54:37Z)
• Neural Artistic Style Transfer with Conditional Adversaria [0.0]
  A neural artistic style transformation model can modify the appearance of a simple image by adding the style of a famous image. In this paper, we present two methods that step toward the style image independent neural style transfer model. Our novel contribution is a unidirectional-GAN model that ensures the Cyclic consistency by the model architecture.
  arXiv  Detail & Related papers  (2023-02-08T04:34:20Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Geometric Style Transfer [74.58782301514053]
  We introduce a neural architecture that supports transfer of geometric style. New architecture runs prior to a network that transfers texture style. Users can input content/style pair as is common, or they can chose to input a content/texture-style/geometry-style triple.
  arXiv  Detail & Related papers  (2020-07-10T16:33:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.