All-to-key Attention for Arbitrary Style Transfer
- URL: http://arxiv.org/abs/2212.04105v2
- Date: Thu, 6 Apr 2023 07:05:13 GMT
- Title: All-to-key Attention for Arbitrary Style Transfer
- Authors: Mingrui Zhu, Xiao He, Nannan Wang, Xiaoyu Wang, Xinbo Gao
- Abstract summary: We propose a novel all-to-key attention mechanism -- each position of content features is matched to stable key positions of style features.
The resultant module, dubbed StyA2K, shows extraordinary performance in preserving the semantic structure and rendering consistent style patterns.
- Score: 98.83954812536521
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Attention-based arbitrary style transfer studies have shown promising
performance in synthesizing vivid local style details. They typically use the
all-to-all attention mechanism -- each position of content features is fully
matched to all positions of style features. However, all-to-all attention tends
to generate distorted style patterns and has quadratic complexity, limiting the
effectiveness and efficiency of arbitrary style transfer. In this paper, we
propose a novel all-to-key attention mechanism -- each position of content
features is matched to stable key positions of style features -- that is more
in line with the characteristics of style transfer. Specifically, it integrates
two newly proposed attention forms: distributed and progressive attention.
Distributed attention assigns attention to key style representations that
depict the style distribution of local regions; Progressive attention pays
attention from coarse-grained regions to fine-grained key positions. The
resultant module, dubbed StyA2K, shows extraordinary performance in preserving
the semantic structure and rendering consistent style patterns. Qualitative and
quantitative comparisons with state-of-the-art methods demonstrate the superior
performance of our approach.
Related papers
- UniVST: A Unified Framework for Training-free Localized Video Style Transfer [66.69471376934034]
This paper presents UniVST, a unified framework for localized video style transfer.
It operates without the need for training, offering a distinct advantage over existing methods that transfer style across entire videos.
arXiv Detail & Related papers (2024-10-26T05:28:02Z) - ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps.
We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
arXiv Detail & Related papers (2024-08-10T08:53:41Z) - TSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style
Transfer [22.16475032434281]
Artistic style transfer aims to create new artistic images by rendering a given photograph with the target artistic style.
Existing methods learn styles simply based on global statistics or local patches, lacking careful consideration of the drawing process in practice.
We propose a Two-Stage Statistics-Aware Transformation (TSSAT) module, which first builds the global style foundation by aligning the global statistics of content and style features.
To further enhance both content and style representations, we introduce two novel losses: an attention-based content loss and a patch-based style loss.
arXiv Detail & Related papers (2023-09-12T07:02:13Z) - AesPA-Net: Aesthetic Pattern-Aware Style Transfer Networks [28.136463099603564]
We focus on enhancing the attention mechanism and capturing the rhythm of patterns that organize the style.
Based on the pattern repeatability, we propose Aesthetic Pattern-Aware style transfer Networks (AesPA-Net)
In addition, we propose a novel self-supervisory task to encourage the attention mechanism to learn precise and meaningful semantic correspondence.
arXiv Detail & Related papers (2023-07-19T02:26:20Z) - A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive
Learning [84.8813842101747]
Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework.
We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature.
Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
arXiv Detail & Related papers (2023-03-09T04:35:00Z) - Learning Graph Neural Networks for Image Style Transfer [131.73237185888215]
State-of-the-art parametric and non-parametric style transfer approaches are prone to either distorted local style patterns due to global statistics alignment, or unpleasing artifacts resulting from patch mismatching.
In this paper, we study a novel semi-parametric neural style transfer framework that alleviates the deficiency of both parametric and non-parametric stylization.
arXiv Detail & Related papers (2022-07-24T07:41:31Z) - Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning.
Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
arXiv Detail & Related papers (2022-05-19T13:11:24Z) - Consistent Style Transfer [23.193302706359464]
Recently, attentional arbitrary style transfer methods have been proposed to achieve fine-grained results.
We propose the progressive attentional manifold alignment (PAMA) to alleviate this problem.
We show that PAMA achieves state-of-the-art performance while avoiding the inconsistency of semantic regions.
arXiv Detail & Related papers (2022-01-06T20:19:35Z) - AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer [44.08659730413871]
We propose a novel attention and normalization module, named Adaptive Attention Normalization (AdaAttN)
Specifically, spatial attention score is learnt from both shallow and deep features of content and style images.
Per-point weighted statistics are calculated by regarding a style feature point as a distribution of attention-weighted output of all style feature points.
Finally, the content feature is normalized so that they demonstrate the same local feature statistics as the calculated per-point weighted style feature statistics.
arXiv Detail & Related papers (2021-08-08T14:26:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.