StyleVAR: Controllable Image Style Transfer via Visual Autoregressive Modeling

• URL: http://arxiv.org/abs/2604.21052v1
• Date: Wed, 22 Apr 2026 19:52:35 GMT
• Title: StyleVAR: Controllable Image Style Transfer via Visual Autoregressive Modeling
• Abstract summary: We formulate style transfer as conditional discrete sequence modeling in a learned latent space.<n>We introduce a blended cross-attention mechanism in which the evolving target representation attends to its own history.<n>Style VAR consistently outperforms an AdaIN baseline on Style Loss, Content Loss, LPIPS, SSIM, DreamSim, and CLIP similarity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We build on the Visual Autoregressive Modeling (VAR) framework and formulate style transfer as conditional discrete sequence modeling in a learned latent space. Images are decomposed into multi-scale representations and tokenized into discrete codes by a VQ-VAE; a transformer then autoregressively models the distribution of target tokens conditioned on style and content tokens. To inject style and content information, we introduce a blended cross-attention mechanism in which the evolving target representation attends to its own history, while style and content features act as queries that decide which aspects of this history to emphasize. A scale-dependent blending coefficient controls the relative influence of style and content at each stage, encouraging the synthesized representation to align with both the content structure and the style texture without breaking the autoregressive continuity of VAR. We train StyleVAR in two stages from a pretrained VAR checkpoint: supervised fine-tuning on a large triplet dataset of content--style--target images, followed by reinforcement fine-tuning with Group Relative Policy Optimization (GRPO) against a DreamSim-based perceptual reward, with per-action normalization weighting to rebalance credit across VAR's multi-scale hierarchy. Across three benchmarks spanning in-, near-, and out-of-distribution regimes, StyleVAR consistently outperforms an AdaIN baseline on Style Loss, Content Loss, LPIPS, SSIM, DreamSim, and CLIP similarity, and the GRPO stage yields further gains over the SFT checkpoint, most notably on the reward-aligned perceptual metrics. Qualitatively, the method transfers texture while maintaining semantic structure, especially for landscapes and architectural scenes, while a generalization gap on internet images and difficulty with human faces highlight the need for better content diversity and stronger structural priors.

Related papers

• Capacity-Controlled Multi-View Stylization of 3D Gaussian Splatting [66.3033154284523]
  We propose a capacity-controlled framework for multi-view stylization of 3DGS.<n>By introducing explicit column-capacity constraints with tunable strength, our formulation mitigates many-to-one matching.<n>We also incorporate a novel cross-view matching guidance to constrain correspondences between scene content and style patterns.
  arXiv  Detail & Related papers  (2026-06-25T08:36:50Z)
• SAGE: Style-Adaptive Generalization for Privacy-Constrained Semantic Segmentation Across Domains [13.393232074517387]
  textbfSAGE improves the generalization of frozen models under privacy constraints.<n>We first utilize style transfer to construct a diverse style representation of the source domain.<n>Then, the model adaptively fuses these style cues according to the visual context of each input, forming a dynamic prompt.
  arXiv  Detail & Related papers  (2025-12-02T03:20:22Z)
• IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction [77.06211178777939]
  IAR2 is an advanced autoregressive framework that enables a hierarchical semantic-detail synthesis process.<n>We show that IAR2 sets a new state-of-the-art for autoregressive image generation, achieving a FID of 1.50 on ImageNet.
  arXiv  Detail & Related papers  (2025-10-08T12:08:21Z)
• Neural Scene Designer: Self-Styled Semantic Image Manipulation [67.43125248646653]
  We introduce the Neural Scene Designer (NSD), a novel framework that enables photo-realistic manipulation of user-specified scene regions.<n>NSD ensures both semantic alignment with user intent and stylistic consistency with the surrounding environment.<n>To capture fine-grained style representations, we propose the Progressive Self-style Representational Learning (PSRL) module.
  arXiv  Detail & Related papers  (2025-09-01T11:59:03Z)
• CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models [5.821094777538556]
  Disentangling content and style from a single image, known as content-style decomposition, enables recontextualization of extracted content and stylization of extracted styles.<n>Visual Autoregressive Modeling has emerged as a promising alternative with a next-scale prediction paradigm.
  arXiv  Detail & Related papers  (2025-07-18T14:45:48Z)
• A Spitting Image: Modular Superpixel Tokenization in Vision Transformers [0.0]
  Vision Transformer (ViT) architectures traditionally employ a grid-based approach to tokenization independent of the semantic content of an image. We propose a modular superpixel tokenization strategy which decouples tokenization and feature extraction.
  arXiv  Detail & Related papers  (2024-08-14T17:28:58Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval [119.03470556503942]
  Crossmodal matching problem is typically solved by learning a joint embedding space where semantic content shared between photo and sketch modalities are preserved. An effective model needs to explicitly account for this style diversity, crucially, to unseen user styles. Our model can not only disentangle the cross-modal shared semantic content, but can adapt the disentanglement to any unseen user style as well, making the model truly agnostic.
  arXiv  Detail & Related papers  (2021-03-29T15:44:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.