HiTokSR: A Coarse-to-Fine Tokenizer with Hierarchical Codebooks for High-Fidelity Real-World Image Super-Resolution

• URL: http://arxiv.org/abs/2606.01157v1
• Date: Sun, 31 May 2026 11:02:58 GMT
• Title: HiTokSR: A Coarse-to-Fine Tokenizer with Hierarchical Codebooks for High-Fidelity Real-World Image Super-Resolution
• Abstract summary: We present HiTokSR, a hierarchical token prediction framework.<n>We show that HiTokSR achieves state-of-the-art performance in both quality and reconstruction fidelity.
• Score: 2.4519602914420475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vector-quantized (VQ) generative models have shown promising results in real-world image super-resolution (Real-ISR). However, existing methods typically rely on a monolithic latent space that entangles low-frequency structures with high-frequency textures. This entanglement forces a single codebook to capture a combinatorially complex set of structure-texture pairings, which constrains representational capacity and limits codebook utilization. To address this issue, we present HiTokSR, a hierarchical token prediction framework. Instead of using a single codebook, HiTokSR partitions the latent space along the channel dimension into frequency-aware groups, quantizing each with an independent sub-codebook. This coarse-to-fine design disentangles global structures from fine details, enhancing combinatorial expressiveness while circumventing the optimization instability of high-dimensional nearest-neighbor lookups. To further improve semantic consistency, our generator integrates priors from a vision foundation model via adaptive feature modulation, multi-scale class tokens, and a representation alignment loss. Additionally, we introduce an index-level perturbation strategy during decoder fine-tuning to bridge the train-test discrepancy in discrete token prediction. Extensive experiments on real-world benchmarks demonstrate that HiTokSR achieves state-of-the-art performance in both perceptual quality and reconstruction fidelity.

Related papers

• Generalized Synthetic Image Detection with Enhanced RGB-Noise Representation Learning [33.97020349697987]
  We propose RNSIDNet, a novel forensic framework that achieves robust detection through enhanced RGB-Noise representation learning.<n>Specifically, our method employs a dual-branch architecture where global RGB semantics, extracted by an attention-refined CLIP backbone, dynamically modulate high-frequency noise artifacts captured by Bayar convolutions via a Feature-wise Linear Modulation (FiLM) module.<n>To further enhance the learned representations, we design a Hard Sample-aware Contrastive Learning (HSCL) strategy.
  arXiv  Detail & Related papers  (2026-07-07T14:52:22Z)
• Unfolding 3D Gaussian Splatting via Iterative Gaussian Synopsis [48.743921535513635]
  Iterative Gaussian Synopsis is a novel framework for compact and progressive rendering through a top-down "unfolding" scheme.<n>Our approach begins with a full-resolution 3DGS model and iteratively derives coarser LODs using an adaptive, learnable mask-based pruning mechanism.<n>This combination produces a compact yet expressive feature representation, designed to minimize redundancy and support efficient, level-specific adaptation.
  arXiv  Detail & Related papers  (2026-04-13T16:28:35Z)
• SNCE: Geometry-Aware Supervision for Scalable Discrete Image Generation [64.29376407025768]
  Neighbor Cross Entropy Minimization (SNCE) is a novel training objective designed to address the optimization challenges of large-codebook discrete image generators.<n>We conduct experiments across class-conditional ImageNet-256 generation, large-scale text-to-image synthesis, and image editing tasks.<n>Results show that SNCE significantly improves convergence speed and overall generation quality compared to standard cross-entropy objectives.
  arXiv  Detail & Related papers  (2026-03-16T11:44:11Z)
• StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models [98.72926158261937]
  We propose a training-free token pruning framework for Visual AutoRegressive models.<n>We employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information.<n>To maintain valid next-scale prediction under sparse tokens, we introduce a nearest neighbor feature propagation strategy.
  arXiv  Detail & Related papers  (2026-03-02T11:35:05Z)
• VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction [83.50898344094153]
  VQRAE produces Continuous semantic features for image understanding and tokens for visual generation within a unified tokenizer.<n>Design enables negligible semantic information for maintaining the ability of multimodal understanding, discrete tokens.<n>VQRAE presents competitive performance on several benchmarks of visual understanding, generation and reconstruction.
  arXiv  Detail & Related papers  (2025-11-28T17:26:34Z)
• HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning [66.99487505369254]
  HiCoGen is built upon a novel Chain of Synthesis paradigm.<n>It decomposes complex prompts into minimal semantic units.<n>It then synthesizes these units iteratively, where the image generated in each step provides crucial visual context for the next.<n>Experiments show our approach significantly outperforms existing methods in both concept coverage and compositional accuracy.
  arXiv  Detail & Related papers  (2025-11-25T06:24:25Z)
• IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction [77.06211178777939]
  IAR2 is an advanced autoregressive framework that enables a hierarchical semantic-detail synthesis process.<n>We show that IAR2 sets a new state-of-the-art for autoregressive image generation, achieving a FID of 1.50 on ImageNet.
  arXiv  Detail & Related papers  (2025-10-08T12:08:21Z)
• Cross-Layer Discrete Concept Discovery for Interpreting Language Models [13.842670153893977]
  Cross-layer VQ-VAE is a framework that uses vector quantization to map representations across layers.<n>Our approach uniquely combines top-k temperature-based sampling during quantization with EMA codebook updates.
  arXiv  Detail & Related papers  (2025-06-24T22:43:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.