QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity
- URL: http://arxiv.org/abs/2212.10431v2
- Date: Mon, 5 Jun 2023 20:20:08 GMT
- Title: QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity
- Authors: Siyu Huang, Jie An, Donglai Wei, Jiebo Luo, Hanspeter Pfister
- Abstract summary: We propose a new style transfer framework called QuantArt for high visual-fidelity stylization.
Our framework achieves significantly higher visual fidelity compared with the existing style transfer methods.
- Score: 94.5479418998225
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The mechanism of existing style transfer algorithms is by minimizing a hybrid
loss function to push the generated image toward high similarities in both
content and style. However, this type of approach cannot guarantee visual
fidelity, i.e., the generated artworks should be indistinguishable from real
ones. In this paper, we devise a new style transfer framework called QuantArt
for high visual-fidelity stylization. QuantArt pushes the latent representation
of the generated artwork toward the centroids of the real artwork distribution
with vector quantization. By fusing the quantized and continuous latent
representations, QuantArt allows flexible control over the generated artworks
in terms of content preservation, style similarity, and visual fidelity.
Experiments on various style transfer settings show that our QuantArt framework
achieves significantly higher visual fidelity compared with the existing style
transfer methods.
Related papers
- ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps.
We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
arXiv Detail & Related papers (2024-08-10T08:53:41Z) - CreativeSynth: Creative Blending and Synthesis of Visual Arts based on
Multimodal Diffusion [74.44273919041912]
Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images.
However, adapting these models for artistic image editing presents two significant challenges.
We build the innovative unified framework Creative Synth, which is based on a diffusion model with the ability to coordinate multimodal inputs.
arXiv Detail & Related papers (2024-01-25T10:42:09Z) - StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized
Tokenizer of a Large-Scale Generative Model [64.26721402514957]
We propose StylerDALLE, a style transfer method that uses natural language to describe abstract art styles.
Specifically, we formulate the language-guided style transfer task as a non-autoregressive token sequence translation.
To incorporate style information, we propose a Reinforcement Learning strategy with CLIP-based language supervision.
arXiv Detail & Related papers (2023-03-16T12:44:44Z) - The Stable Artist: Steering Semantics in Diffusion Latent Space [17.119616029527744]
We present the Stable Artist, an image editing approach enabling fine-grained control of the image generation process.
The main component is semantic guidance (SEGA) which steers the diffusion process along variable numbers of semantic directions.
SEGA enables probing of latent spaces to gain insights into the representation of concepts learned by the model.
arXiv Detail & Related papers (2022-12-12T16:21:24Z) - Line Search-Based Feature Transformation for Fast, Stable, and Tunable
Content-Style Control in Photorealistic Style Transfer [26.657485176782934]
Photorealistic style transfer is the task of synthesizing a realistic-looking image when adapting the content from one image to appear in the style of another image.
Modern models embed a transformation that fuses features describing the content image and style image and then decodes the resulting feature into a stylized image.
We introduce a general-purpose transformation that enables controlling the balance between how much content is preserved and the strength of the infused style.
arXiv Detail & Related papers (2022-10-12T08:05:49Z) - CCPL: Contrastive Coherence Preserving Loss for Versatile Style Transfer [58.020470877242865]
We devise a universally versatile style transfer method capable of performing artistic, photo-realistic, and video style transfer jointly.
We make a mild and reasonable assumption that global inconsistency is dominated by local inconsistencies and devise a generic Contrastive Coherence Preserving Loss (CCPL) applied to local patches.
CCPL can preserve the coherence of the content source during style transfer without degrading stylization.
arXiv Detail & Related papers (2022-07-11T12:09:41Z) - Interactive Style Transfer: All is Your Palette [74.06681967115594]
We propose a drawing-like interactive style transfer (IST) method, by which users can interactively create a harmonious-style image.
Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image.
arXiv Detail & Related papers (2022-03-25T06:38:46Z) - Fine-Grained Control of Artistic Styles in Image Generation [24.524863555822837]
generative models and adversarial training have enabled artificially generating artworks in various artistic styles.
We propose to capture the continuous spectrum of styles and apply it to a style generation task.
Our method can be used with common generative adversarial networks (such as StyleGAN)
arXiv Detail & Related papers (2021-10-19T21:51:52Z) - Anisotropic Stroke Control for Multiple Artists Style Transfer [36.92721585146738]
Stroke Control Multi-Artist Style Transfer framework is developed.
Anisotropic Stroke Module (ASM) endows the network with the ability of adaptive semantic-consistency among various styles.
In contrast to the single-scale conditional discriminator, our discriminator is able to capture multi-scale texture clue.
arXiv Detail & Related papers (2020-10-16T05:32:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.