TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders

• URL: http://arxiv.org/abs/2604.07340v1
• Date: Wed, 08 Apr 2026 17:53:52 GMT
• Title: TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders
• Abstract summary: We propose TC-AE, a ViT-based architecture for deep compression autoencoders.<n>We decompose token-to-latent compression into two stages, reducing structural information loss.<n>We enhance the semantic structure of image tokens via joint self-supervised training, leading to more generative-friendly latents.
• Score: 51.71228803075235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose TC-AE, a ViT-based architecture for deep compression autoencoders. Existing methods commonly increase the channel number of latent representations to maintain reconstruction quality under high compression ratios. However, this strategy often leads to latent representation collapse, which degrades generative performance. Instead of relying on increasingly complex architectures or multi-stage training schemes, TC-AE addresses this challenge from the perspective of the token space, the key bridge between pixels and image latents, through two complementary innovations: Firstly, we study token number scaling by adjusting the patch size in ViT under a fixed latent budget, and identify aggressive token-to-latent compression as the key factor that limits effective scaling. To address this issue, we decompose token-to-latent compression into two stages, reducing structural information loss and enabling effective token number scaling for generation. Secondly, to further mitigate latent representation collapse, we enhance the semantic structure of image tokens via joint self-supervised training, leading to more generative-friendly latents. With these designs, TC-AE achieves substantially improved reconstruction and generative performance under deep compression. We hope our research will advance ViT-based tokenizer for visual generation.

Related papers

• RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT [0.014257559724536567]
  Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity.<n>We propose a depth-aware token reduction framework that adapts reduction strategies to the layer-wise characteristics of token representations.<n>Our framework provides a training-free template for optimizing vision models by aligning reduction strategies with hierarchical feature evolution.
  arXiv  Detail & Related papers  (2026-06-06T13:13:02Z)
• SFTok: Bridging the Performance Gap in Discrete Tokenizers [72.9996757048065]
  We propose textbfSFTok, a discrete tokenizer that incorporates a multi-step iterative mechanism for precise reconstruction.<n>At a high compression rate of only 64 tokens per image, SFTok achieves state-of-the-art reconstruction quality on ImageNet.
  arXiv  Detail & Related papers  (2025-12-18T18:59:04Z)
• Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI [26.45869748408205]
  Token compression techniques have emerged as powerful tools for Vision Transformer (ViT) inference in computer vision.<n>We present the first systematic taxonomy and comparative study of token compression methods.<n>Our experiments reveal that while token compression methods are effective for general-purpose ViTs, they often underperform when directly applied to compact designs.
  arXiv  Detail & Related papers  (2025-07-13T16:26:05Z)
• H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Models [97.45170082949552]
  Autoencoder (AE) is the key to the success of latent diffusion models for image and video generation.<n>H3AE achieves ultra-high compression ratios and real-time decoding speed on GPU and mobile.
  arXiv  Detail & Related papers  (2025-04-14T17:59:06Z)
• GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation [81.58846231702026]
  We introduce GigaTok, the first approach to improve image reconstruction, generation, and representation learning when scaling visual tokenizers.<n>We identify the growing complexity of latent space as the key factor behind the reconstruction vs. generation dilemma.<n>By scaling to $bf3 space billion$ parameters, GigaTok achieves state-of-the-art performance in reconstruction, downstream AR generation, and downstream AR representation quality.
  arXiv  Detail & Related papers  (2025-04-11T17:59:58Z)
• Pathology Image Compression with Pre-trained Autoencoders [52.208181380986524]
  Whole Slide Images in digital histopathology pose significant storage, transmission, and computational efficiency challenges.<n>Standard compression methods, such as JPEG, reduce file sizes but fail to preserve fine-grained phenotypic details critical for downstream tasks.<n>In this work, we repurpose autoencoders (AEs) designed for Latent Diffusion Models as an efficient learned compression framework for pathology images.
  arXiv  Detail & Related papers  (2025-03-14T17:01:17Z)
• Epsilon-VAE: Denoising as Visual Decoding [61.29255979767292]
  We propose denoising as decoding, shifting from single-step reconstruction to iterative refinement.<n>Specifically, we replace the decoder with a diffusion process that iteratively refines noise to recover the original image.<n>By adopting iterative reconstruction through diffusion, our autoencoder, namely Epsilon-VAE, achieves high reconstruction quality.
  arXiv  Detail & Related papers  (2024-10-05T08:27:53Z)
• Generalized Nested Latent Variable Models for Lossy Coding applied to Wind Turbine Scenarios [14.48369551534582]
  A learning-based approach seeks to minimize the compromise between compression rate and reconstructed image quality. A successful technique consists in introducing a deep hyperprior that operates within a 2-level nested latent variable model. This paper extends this concept by designing a generalized L-level nested generative model with a Markov chain structure.
  arXiv  Detail & Related papers  (2024-06-10T11:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.