Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens

• URL: http://arxiv.org/abs/2603.19232v1
• Date: Thu, 19 Mar 2026 17:59:55 GMT
• Title: Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens
• Abstract summary: We presentCub Discrete Diffusion (CubiD), the first discrete generation model for high-dimensional representations.<n>CubiD performs fine-grained masking throughout the high-dimensional discrete representation.<n>On ImageNet-256, CubiD achieves state-of-the-art discrete generation with strong scaling behavior from 900M to 3.7B parameters.
• Score: 88.42820935044021
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual generation with discrete tokens has gained significant attention as it enables a unified token prediction paradigm shared with language models, promising seamless multimodal architectures. However, current discrete generation methods remain limited to low-dimensional latent tokens (typically 8-32 dims), sacrificing the semantic richness essential for understanding. While high-dimensional pretrained representations (768-1024 dims) could bridge this gap, their discrete generation poses fundamental challenges. In this paper, we present Cubic Discrete Diffusion (CubiD), the first discrete generation model for high-dimensional representations. CubiD performs fine-grained masking throughout the high-dimensional discrete representation -- any dimension at any position can be masked and predicted from partial observations. This enables the model to learn rich correlations both within and across spatial positions, with the number of generation steps fixed at $T$ regardless of feature dimensionality, where $T \ll hwd$. On ImageNet-256, CubiD achieves state-of-the-art discrete generation with strong scaling behavior from 900M to 3.7B parameters. Crucially, we validate that these discretized tokens preserve original representation capabilities, demonstrating that the same discrete tokens can effectively serve both understanding and generation tasks. We hope this work will inspire future research toward unified multimodal architectures. Code is available at: https://github.com/YuqingWang1029/CubiD.

Related papers

• Where Does Generative Difficulty Reside? An Empirical Study of Target Representations [62.54876287800644]
  The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface.<n>We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model.<n>Our results show that compression, reconstruction fidelity, token dimensionality, and visible semantic clustering do not individually predict generative behavior.
  arXiv  Detail & Related papers  (2026-08-01T12:32:31Z)
• Kelix Technical Report [86.64551727600104]
  We present Kelix, a fully discrete autoregressive unified model that closes the understanding gap between discrete and continuous visual representations.<n>Recent work has explored discrete visual tokenization to enable fully autoregressive multimodal modeling.
  arXiv  Detail & Related papers  (2026-02-10T14:48:26Z)
• DINO-Tok: Adapting DINO for Visual Tokenizers [52.194754463297706]
  DINO-Tok is a visual tokenizer that unifies hierarchical representations into an information-complete latent space.<n>On ImageNet, DINO-Tok achieves state-of-the-art reconstruction performance, reaching 28.54 PSNR for autoencoding and 23.98 PSNR for VQ-based modeling.
  arXiv  Detail & Related papers  (2025-11-25T18:00:00Z)
• Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis [79.98107530577576]
  DisCon is a novel framework that reinterprets discrete tokens as conditional signals rather than generation targets.<n>DisCon achieves a gFID score of 1.38 on ImageNet 256$times $256 generation, outperforming state-of-the-art autoregressive approaches by a clear margin.
  arXiv  Detail & Related papers  (2025-07-02T14:33:52Z)
• Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation [85.82112629564942]
  We propose TokenBridge, which maintains the strong representation capacity of continuous tokens while preserving the modeling simplicity of discrete tokens.<n>We introduce a dimension-wise quantization strategy that independently discretizes each feature dimension, paired with a lightweight autoregressive prediction mechanism.<n>Our approach achieves reconstruction and generation quality on par with continuous methods while using standard categorical prediction.
  arXiv  Detail & Related papers  (2025-03-20T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.