Autoregressive Styled Text Image Generation, but Make it Reliable

• URL: http://arxiv.org/abs/2510.23240v1
• Date: Mon, 27 Oct 2025 11:54:23 GMT
• Title: Autoregressive Styled Text Image Generation, but Make it Reliable
• Authors: Carmine Zaccagnino, Fabio Quattrini, Vittorio Pippi, Silvia Cascianelli, Alessio Tonioni, Rita Cucchiara,
• Abstract summary: This work is dedicated to developing strategies that reproduce the characteristics of a given writer, with promising results in terms of style fidelity and generalization achieved by the recently proposed Autoregressive Transformer paradigm for HTG.<n>In this work, we rethink the autoregressive by framing HTG as a multimodal prompt-conditioned generation task, tackling the content controllability issues by introducing special input tokens for better alignment with the visual ones.
• Score: 51.09340470015673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating faithful and readable styled text images (especially for Styled Handwritten Text generation - HTG) is an open problem with several possible applications across graphic design, document understanding, and image editing. A lot of research effort in this task is dedicated to developing strategies that reproduce the stylistic characteristics of a given writer, with promising results in terms of style fidelity and generalization achieved by the recently proposed Autoregressive Transformer paradigm for HTG. However, this method requires additional inputs, lacks a proper stop mechanism, and might end up in repetition loops, generating visual artifacts. In this work, we rethink the autoregressive formulation by framing HTG as a multimodal prompt-conditioned generation task, and tackle the content controllability issues by introducing special textual input tokens for better alignment with the visual ones. Moreover, we devise a Classifier-Free-Guidance-based strategy for our autoregressive model. Through extensive experimental validation, we demonstrate that our approach, dubbed Eruku, compared to previous solutions requires fewer inputs, generalizes better to unseen styles, and follows more faithfully the textual prompt, improving content adherence.

Related papers

• Sissi: Zero-shot Style-guided Image Synthesis via Semantic-style Integration [57.02757226679549]
  We introduce a training-free framework that reformulates style-guided synthesis as an in-context learning task.<n>We propose a Dynamic Semantic-Style Integration (DSSI) mechanism that reweights attention between semantic and style visual tokens.<n>Experiments show that our approach achieves high-fidelity stylization with superior semantic-style balance and visual quality.
  arXiv  Detail & Related papers  (2026-01-10T16:01:14Z)
• ScriptViT: Vision Transformer-Based Personalized Handwriting Generation [0.0]
  styled handwriting generation aims to synthesize handwritten text that looks both realistic and aligned with a specific writer's style.<n>We introduce a Vision Transformer-based style encoder that learns global stylistic patterns from multiple reference images.<n>We then integrate these style cues with the target text using a cross-attention mechanism, enabling the system to produce handwritten images that more faithfully reflect the intended style.
  arXiv  Detail & Related papers  (2025-11-23T06:38:23Z)
• Qwen-Image Technical Report [86.46471547116158]
  We present Qwen-Image, an image generation foundation model that achieves significant advances in complex text rendering and precise image editing.<n>We design a comprehensive data pipeline that includes large-scale data collection, filtering, annotation, synthesis, and balancing.<n>Qwen-Image performs exceptionally well in alphabetic languages such as English, and also achieves remarkable progress on more challenging logographic languages like Chinese.
  arXiv  Detail & Related papers  (2025-08-04T11:49:20Z)
• Beyond Words: Advancing Long-Text Image Generation via Multimodal Autoregressive Models [76.68654868991517]
  Long-form text in images, such as paragraphs in slides or documents, remains a major challenge for current generative models.<n>We introduce a novel text-focused, binary tokenizer optimized for capturing detailed scene text features.<n>We develop ModelName, a multimodal autoregressive model that excels in generating high-quality long-text images with unprecedented fidelity.
  arXiv  Detail & Related papers  (2025-03-26T03:44:25Z)
• Zero-Shot Styled Text Image Generation, but Make It Autoregressive [34.09957000751439]
  Styled Handwritten Text Generation (HTG) has recently received attention from the computer vision and document analysis communities.<n>We propose a novel framework for text image generation, dubbed Emuru.<n>Our approach leverages a powerful text image representation model (a variational autoencoder) combined with an autoregressive Transformer.
  arXiv  Detail & Related papers  (2025-03-21T11:56:20Z)
• IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Prompting [71.29100512700064]
  IP-Prompter is a novel training-free TSI generation method.<n>It integrates reference images into generative models, allowing users to seamlessly specify the target theme.<n>Our approach enables diverse applications, including consistent story generation, character design, realistic character generation, and style-guided image generation.
  arXiv  Detail & Related papers  (2025-01-26T19:01:19Z)
• Towards Visual Text Design Transfer Across Languages [49.78504488452978]
  We introduce a novel task of Multimodal Style Translation (MuST-Bench) MuST-Bench is a benchmark designed to evaluate the ability of visual text generation models to perform translation across different writing systems. In response, we introduce SIGIL, a framework for multimodal style translation that eliminates the need for style descriptions.
  arXiv  Detail & Related papers  (2024-10-24T15:15:01Z)
• Masked Generative Story Transformer with Character Guidance and Caption Augmentation [2.1392064955842023]
  Story visualization is a challenging generative vision task, that requires both visual quality and consistency between different frames in generated image sequences. Previous approaches either employ some kind of memory mechanism to maintain context throughout an auto-regressive generation of the image sequence, or model the generation of the characters and their background separately. We propose a completely parallel transformer-based approach, relying on Cross-Attention with past and future captions to achieve consistency.
  arXiv  Detail & Related papers  (2024-03-13T13:10:20Z)
• Visual Style Prompting with Swapping Self-Attention [26.511518230332758]
  We propose a novel approach to produce a diverse range of images while maintaining specific style elements and nuances. During the denoising process, we keep the query from original features while swapping the key and value with those from reference features in the late self-attention layers. Our method demonstrates superiority over existing approaches, best reflecting the style of the references and ensuring that resulting images match the text prompts most accurately.
  arXiv  Detail & Related papers  (2024-02-20T12:51:17Z)
• Handwriting Transformers [98.3964093654716]
  We propose a transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement and global and local writing style patterns. The proposed HWT captures the long and short range relationships within the style examples through a self-attention mechanism. Our proposed HWT generates realistic styled handwritten text images and significantly outperforms the state-of-the-art demonstrated.
  arXiv  Detail & Related papers  (2021-04-08T17:59:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.