Learning Dynamic Style Kernels for Artistic Style Transfer
- URL: http://arxiv.org/abs/2304.00414v2
- Date: Fri, 14 Apr 2023 21:27:27 GMT
- Title: Learning Dynamic Style Kernels for Artistic Style Transfer
- Authors: Wenju Xu and Chengjiang Long and Yongwei Nie
- Abstract summary: We propose a new scheme that learns em spatially adaptive kernels for per-pixel stylization.
Our proposed method outperforms state-of-the-art methods and exhibits superior performance in terms of visual quality and efficiency.
- Score: 26.19086645743083
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Arbitrary style transfer has been demonstrated to be efficient in artistic
image generation. Previous methods either globally modulate the content feature
ignoring local details, or overly focus on the local structure details leading
to style leakage. In contrast to the literature, we propose a new scheme
\textit{``style kernel"} that learns {\em spatially adaptive kernels} for
per-pixel stylization, where the convolutional kernels are dynamically
generated from the global style-content aligned feature and then the learned
kernels are applied to modulate the content feature at each spatial position.
This new scheme allows flexible both global and local interactions between the
content and style features such that the wanted styles can be easily
transferred to the content image while at the same time the content structure
can be easily preserved. To further enhance the flexibility of our style
transfer method, we propose a Style Alignment Encoding (SAE) module
complemented with a Content-based Gating Modulation (CGM) module for learning
the dynamic style kernels in focusing regions. Extensive experiments strongly
demonstrate that our proposed method outperforms state-of-the-art methods and
exhibits superior performance in terms of visual quality and efficiency.
Related papers
- ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model [73.95608242322949]
Stylized Text-to-Image Generation (STIG) aims to generate images from text prompts and style reference images.
We present ArtWeaver, a novel framework that leverages pretrained Stable Diffusion to address challenges such as misinterpreted styles and inconsistent semantics.
arXiv Detail & Related papers (2024-05-24T07:19:40Z) - DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization [43.67213274161226]
Source-Free Domain Generalization (SFDG) aims to develop a model that works for unseen target domains without relying on any source domain.
Research in SFDG primarily bulids upon the existing knowledge of large-scale vision-language models.
We introduce Dynamic PromptStyler (DPStyler), comprising Style Generation and Style Removal modules.
arXiv Detail & Related papers (2024-03-25T12:31:01Z) - A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive
Learning [84.8813842101747]
Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework.
We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature.
Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
arXiv Detail & Related papers (2023-03-09T04:35:00Z) - Learning Graph Neural Networks for Image Style Transfer [131.73237185888215]
State-of-the-art parametric and non-parametric style transfer approaches are prone to either distorted local style patterns due to global statistics alignment, or unpleasing artifacts resulting from patch mismatching.
In this paper, we study a novel semi-parametric neural style transfer framework that alleviates the deficiency of both parametric and non-parametric stylization.
arXiv Detail & Related papers (2022-07-24T07:41:31Z) - Diagonal Attention and Style-based GAN for Content-Style Disentanglement
in Image Generation and Translation [34.24876359759408]
We present a novel hierarchical adaptive Diagonal spatial ATtention layers to manipulate the spatial contents from styles in a hierarchical manner.
Our method enables coarse-to-fine level disentanglement of spatial contents and styles.
Our generator can be easily integrated into the GAN inversion framework so that the content and style of translated images can be flexibly controlled.
arXiv Detail & Related papers (2021-03-30T08:00:13Z) - Arbitrary Video Style Transfer via Multi-Channel Correlation [84.75377967652753]
We propose Multi-Channel Correction network (MCCNet) to fuse exemplar style features and input content features for efficient style transfer.
MCCNet works directly on the feature space of style and content domain where it learns to rearrange and fuse style features based on similarity with content features.
The outputs generated by MCC are features containing the desired style patterns which can further be decoded into images with vivid style textures.
arXiv Detail & Related papers (2020-09-17T01:30:46Z) - Arbitrary Style Transfer via Multi-Adaptation Network [109.6765099732799]
A desired style transfer, given a content image and referenced style painting, would render the content image with the color tone and vivid stroke patterns of the style painting.
A new disentanglement loss function enables our network to extract main style patterns and exact content structures to adapt to various input images.
arXiv Detail & Related papers (2020-05-27T08:00:22Z) - Manifold Alignment for Semantically Aligned Style Transfer [61.1274057338588]
We make a new assumption that image features from the same semantic region form a manifold and an image with multiple semantic regions follows a multi-manifold distribution.
Based on this assumption, the style transfer problem is formulated as aligning two multi-manifold distributions.
The proposed framework allows semantically similar regions between the output and the style image share similar style patterns.
arXiv Detail & Related papers (2020-05-21T16:52:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.