Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering

• URL: http://arxiv.org/abs/2606.01911v1
• Date: Mon, 01 Jun 2026 08:47:17 GMT
• Title: Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering
• Abstract summary: Visual Autoregressive (AR) models generate images by predicting discrete tokens that are decoded by a visual tokenizer.<n>Despite demonstrating strong overall image generation ability, they still underperform on text rendering with blur strokes and disrupt letter shapes.<n>We propose the Residual Decoder Adapter(RDA) that upgrades an existing tokenizer post-hoc without changing its token space.
• Score: 92.43552212966732
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual Autoregressive (AR) models generate images by predicting discrete tokens that are decoded by a visual tokenizer. Despite demonstrating strong overall image generation ability, they still underperform on text rendering with blur strokes and disrupt letter shapes. In this work, we trace this limitation to the visual tokenizer, which struggles to reconstruct fine-grained detail. Improving the tokenizer is straightforward but expensive, as it necessitates retraining both the tokenizer and the AR model. Can we improve text rendering performance of AR models without retraining the existing tokenizer and AR model? To achieve this, we propose the Residual Decoder Adapter(RDA) that upgrades an existing tokenizer post-hoc without changing its token space. Specifically, it refines the decoder output of the visual tokenizer by introducing two novel components: (i) a paired codebook that shares the token distribution with the original one; (ii) a parallel branch to learn the tiny differences (residual) between the reconstructed image and the ground-truth images in the pixel space. This residual design allows us to enhance the tokenizer non-invasively while preserving compatibility with prior AR models. RDA substantially improves text rendering significantly by a large margin. For instance, we boost finetuned Janus-Pro OCR accuracy rises from 24.52% to 58.26% (TextVisionBlend), from 12.75% to 36.81% (StyledTextSynth) on competitive TextAtlas benchmark. The code is available at https://github.com/CSU-JPG/RDA

Related papers

• Improving Flexible Image Tokenizers for Autoregressive Image Generation [53.238708824055664]
  textbfReToK is a flexible tokenizer with underlineRedundant underlineToken Padding and Hierarchical Semantic Regularization.<n>Our method achieves superior generation performance compared with both flexible and fixed-length tokenizers.
  arXiv  Detail & Related papers  (2026-01-04T14:11:45Z)
• VA-$π$: Variational Policy Alignment for Pixel-Aware Autoregressive Generation [65.22452273252428]
  VA-$$ is a post-training framework to optimize autoregressive visual generation.<n>It unifies pixel reconstruction and autoregressive modeling.<n>It reduces FID from 14.36 to 7.65 and improves IS from 86.55 to 116.70 on LlamaGen-XXL.
  arXiv  Detail & Related papers  (2025-12-22T18:54:30Z)
• REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularization [130.46612643194973]
  reAR is a simple training strategy introducing a token-wise regularization objective.<n>On ImageNet, it reduces gFID from 3.02 to 1.86 and improves IS to 316.9 using a standardization-based tokenizer.<n>When applied to advanced tokenizers, it achieves a gFID of 1.42 with only 177M parameters, matching the performance with larger state-of-the-art diffusion models (675M)
  arXiv  Detail & Related papers  (2025-10-06T02:48:13Z)
• Hita: Holistic Tokenizer for Autoregressive Image Generation [56.81871174745175]
  We introduce textitHita, a novel image tokenizer for autoregressive (AR) image generation.<n>It introduces a holistic-to-local tokenization scheme with learnable holistic queries and local patch tokens.
  arXiv  Detail & Related papers  (2025-07-03T06:44:26Z)
• Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models [92.18057318458528]
  Token-Shuffle is a novel method that reduces the number of image tokens in Transformer.<n>Our strategy requires no additional pretrained text-encoder and enables MLLMs to support extremely high-resolution image synthesis.<n>In GenAI-benchmark, our 2.7B model achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen by 0.18 and diffusion models LDM by 0.15.
  arXiv  Detail & Related papers  (2025-04-24T17:59:56Z)
• Spot the Error: Non-autoregressive Graphic Layout Generation with Wireframe Locator [37.65937921431886]
  We propose a learning-based locator to detect erroneous tokens which takes the wireframe image rendered from the generated layout sequence as input. We show that it serves as a complementary modality to the element sequence in object space and contributes greatly to the overall performance.
  arXiv  Detail & Related papers  (2024-01-29T18:13:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.