Hierarchical Image Tokenization for Multi-Scale Image Super Resolution

• URL: http://arxiv.org/abs/2605.14891v1
• Date: Thu, 14 May 2026 14:35:51 GMT
• Title: Hierarchical Image Tokenization for Multi-Scale Image Super Resolution
• Abstract summary: We introduce a multi-scale Image Super Resolution (ISR) method building on recent advances in Visual Auto-Regressive ( VAR) modeling.<n>We introduce two novel components to the VAR training for ISR, aiming at increasing its flexibility and reducing its complexity.<n>Our proposed HIT acts as a strong inductive bias for the VAR training, resulting in a small model that achieves state-of-the-art results without external training data.
• Score: 47.66856101347057
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a multi-scale Image Super Resolution (ISR) method building on recent advances in Visual Auto-Regressive (VAR) modeling. VAR models break image tokenization into additive, gradually increasing scales, using Residual Quantization (RQ), an approach that aligns perfectly with our target ISR task. Previous works taking advantage of this synergy suffer from two main shortcomings. First, due to the limitations in RQ, they only generate images at a predefined fixed scale, failing to map intermediate outputs to the corresponding image scales. They also rely on large backbones or a large corpus of annotated data to achieve better performance. To address both shortcomings, we introduce two novel components to the VAR training for ISR, aiming at increasing its flexibility and reducing its complexity. In particular, we introduce a) a \textbf{Hierarchical Image Tokenization (HIT)} approach that progressively represents images at different scales while enforcing token overlap across scales, and b) a \textbf{Direct Preference Optimization (DPO) regularization term} that, relying solely on the (LR,HR) pair, encourages the transformer to produce the latter over the former. Our proposed HIT acts as a strong inductive bias for the VAR training, resulting in a small model (300M params vs 1B params of VARSR), that achieves state-of-the-art results without external training data, and that delivers multi-scale outputs with a single forward pass.

Related papers

• MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts [41.67455261906954]
  We introduce a scale-aware token-routed Mixture of Experts (MoE) architecture, allowing scale-adaptive expert selection.<n>Unlike naive alignment, we analyse and design a residual feature aggregation scheme tailored to the Visual AutoRegressive modeling paradigm.<n>Our model achieves a superior FID compared to the dense baseline on the ImageNet 256*256 benchmark.
  arXiv  Detail & Related papers  (2026-07-01T03:11:21Z)
• VARestorer: One-Step VAR Distillation for Real-World Image Super-Resolution [76.19751531910039]
  We propose VARestorer, a framework that transforms a pre-trained text-to-image VAR model into a one-step ISR model.<n>VARestorer achieves state-of-the-art performance with 72.32 MUSIQ and 0.7669 CLIPIQA on DIV2K dataset.
  arXiv  Detail & Related papers  (2026-04-23T09:09:36Z)
• VOSR: A Vision-Only Generative Model for Image Super-Resolution [28.65862989487937]
  We propose a Vision-Only generative framework for generative image super-resolution (SR)<n>We first extract semantically rich and spatially grounded features from the LR input using a pretrained vision encoder as visual semantic guidance.<n>Our results, for the first time, show that high-quality generative SR can be achieved without multimodal pretraining.
  arXiv  Detail & Related papers  (2026-04-03T17:50:29Z)
• Multi-scale Image Super Resolution with a Single Auto-Regressive Model [40.77470215283583]
  We tackle Image Super Resolution (ISR) using recent advances in Visual Auto-Regressive ( VAR) modeling.<n>To the best of our knowledge, this is the first time a quantizer is trained to force semantically consistent residuals at different scales.<n>Our model can denoise the LR image and super-resolve at half and full target upscale factors in a single forward pass.
  arXiv  Detail & Related papers  (2025-06-05T13:02:23Z)
• Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment [51.99765487172328]
  Chain-of-zoom (CoZ) is a framework that factorizes SISR into a chain of intermediate scale-states with multi-scale-aware prompts.<n>Because visual cues diminish at high magnifications, we augment each zoom step with multi-scale-aware text prompts generated by a vision-language model (VLM)<n>Experiments show that a standard 4x diffusion SR model wrapped in CoZ attains beyond 256x enlargement with high perceptual quality and fidelity.
  arXiv  Detail & Related papers  (2025-05-24T08:50:08Z)
• FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matching [34.112157859384645]
  We introduce FlowAR, a next scale prediction method featuring a streamlined scale design.<n>This eliminates the need for VAR's intricate multi-scale residual tokenizer.<n>We validate the effectiveness of FlowAR on the challenging ImageNet-256 benchmark.
  arXiv  Detail & Related papers  (2024-12-19T18:59:31Z)
• Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
  Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly. A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work. It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
  arXiv  Detail & Related papers  (2022-09-26T22:22:30Z)
• Towards Lightweight Super-Resolution with Dual Regression Learning [58.98801753555746]
  Deep neural networks have exhibited remarkable performance in image super-resolution (SR) tasks. The SR problem is typically an ill-posed problem and existing methods would come with several limitations. We propose a dual regression learning scheme to reduce the space of possible SR mappings.
  arXiv  Detail & Related papers  (2022-07-16T12:46:10Z)
• LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond [75.37541439447314]
  Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. This paper proposes a linearly-assembled pixel-adaptive regression network (LAPAR) to strike a sweet spot of deep model complexity and resulting SISR quality.
  arXiv  Detail & Related papers  (2021-05-21T15:47:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.