dRAE: Representation Autoencoder with Hyper-Spherical Codes

• URL: http://arxiv.org/abs/2607.22148v1
• Date: Fri, 24 Jul 2026 09:47:32 GMT
• Title: dRAE: Representation Autoencoder with Hyper-Spherical Codes
• Abstract summary: This work aims to discretize the high-dimensional visual representations to bridge the gap with language models.<n>Existing quantization methods suffer from codebook collapse, failing to scale while preserving semantic coherence.<n>We propose Hyper-Spherical Quantization (HSQ), which decouples semantic content from feature magnitude via angular routing.
• Score: 46.51476423608179
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we aim to discretize the high-dimensional visual representations to bridge the gap with language models - a non-trivial challenge, as existing quantization methods suffer from codebook collapse, failing to scale while preserving semantic coherence. We identify the root cause as metric mismatch: standard Euclidean codebook objectives are fundamentally misaligned with the anisotropic geometry of representation space, leading to codebook embeddings with high-variance magnitude scales and uneven angular distributions that hinder scalability. To address this, we propose Hyper-Spherical Quantization (HSQ), which decouples semantic content from feature magnitude via angular routing, preventing code assignment from being dominated by scale rather than meaning. The resulting discrete Representation Autoencoder (dRAE) achieves high-fidelity reconstruction while preserving semantic integrity and supporting scalable codebook budget. Extensive experiments demonstrate consistent performance gains as the vocabulary size scales to 131{,}072, along with 100\% codebook utilization, simplified training pipeline, and strong performance across understanding and generation tasks.

Related papers

• HiTokSR: A Coarse-to-Fine Tokenizer with Hierarchical Codebooks for High-Fidelity Real-World Image Super-Resolution [2.4519602914420475]
  We present HiTokSR, a hierarchical token prediction framework.<n>We show that HiTokSR achieves state-of-the-art performance in both quality and reconstruction fidelity.
  arXiv  Detail & Related papers  (2026-05-31T11:02:58Z)
• Boosting Quantitive and Spatial Awareness for Zero-Shot Object Counting [21.127727991498244]
  Zero-shot object counting (ZSOC) aims to enumerate objects of arbitrary categories specified by text descriptions without requiring visual exemplars.<n>Existing methods often treat counting as a coarse retrieval task, suffering from a lack of fine-grained quantity awareness.<n>We present textbfQICA, a novel framework that synergizes underlinequantity perceptunderlineion with robust spatial underlinecast underlineaggregation.
  arXiv  Detail & Related papers  (2026-03-17T05:32:09Z)
• SNCE: Geometry-Aware Supervision for Scalable Discrete Image Generation [64.29376407025768]
  Neighbor Cross Entropy Minimization (SNCE) is a novel training objective designed to address the optimization challenges of large-codebook discrete image generators.<n>We conduct experiments across class-conditional ImageNet-256 generation, large-scale text-to-image synthesis, and image editing tasks.<n>Results show that SNCE significantly improves convergence speed and overall generation quality compared to standard cross-entropy objectives.
  arXiv  Detail & Related papers  (2026-03-16T11:44:11Z)
• VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction [83.50898344094153]
  VQRAE produces Continuous semantic features for image understanding and tokens for visual generation within a unified tokenizer.<n>Design enables negligible semantic information for maintaining the ability of multimodal understanding, discrete tokens.<n>VQRAE presents competitive performance on several benchmarks of visual understanding, generation and reconstruction.
  arXiv  Detail & Related papers  (2025-11-28T17:26:34Z)
• Surface code scaling on heavy-hex superconducting quantum processors [36.94429692322632]
  We demonstrate subthreshold scaling of a surface-code quantum memory on hardware whose native connectivity does not match the code.<n>We show that DD plays a major role: it suppresses coherent ZZ crosstalk and non-Markovian dephasing.<n>We derive an entanglement fidelity metric that is computed directly from X- and Z-basis logical-error data and provides per-cycle, SPAM-aware bounds.
  arXiv  Detail & Related papers  (2025-10-21T17:37:40Z)
• Exploiting Discriminative Codebook Prior for Autoregressive Image Generation [54.14166700058777]
  token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm.<n>While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited.<n>Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook.<n>We propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means
  arXiv  Detail & Related papers  (2025-08-14T15:00:00Z)
• UniCode$^2$: Cascaded Large-scale Codebooks for Unified Multimodal Understanding and Generation [5.286124283204446]
  We propose UniCode$2$, a cascaded codebook framework enabling large-scale, semantically aligned, and stable visual tokenization.<n>By clustering millions of SigLIP sequence embeddings, we build a 500K-entry codebook that preserves vision-language alignment while expanding capacity.<n>UniCode2 delivers strong performance across diverse benchmarks, demonstrating the viability of scaling visual token spaces without sacrificing stability, semantics, or modularity.
  arXiv  Detail & Related papers  (2025-06-25T07:57:09Z)
• SGC-VQGAN: Towards Complex Scene Representation via Semantic Guided Clustering Codebook [9.993066868670283]
  We introduce SGC-VQGAN through Semantic Online Clustering method to enhance token semantics through Consistent Semantic Learning. Our approach constructs a temporospatially consistent semantic codebook, addressing issues of codebook collapse and imbalanced token semantics.
  arXiv  Detail & Related papers  (2024-09-09T23:12:43Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.