Elastic Token Compression for Pixel-Space Diffusion Transformers

• URL: http://arxiv.org/abs/2608.29281v1
• Date: Sat, 29 Aug 2026 14:06:27 GMT
• Title: Elastic Token Compression for Pixel-Space Diffusion Transformers
• Abstract summary: Region Token Interface (Method) adapts a diffusion model to these tokens, with the region count drawn at random during fine-tuning.<n>Method leads prior reduction methods at matched budgets, matches dense quality at $2.0times$ the speed, and stays close at $2.6times$.
• Score: 64.18588817660013
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural images concentrate their detail in a small fraction of the frame, yet diffusion models spend a full token on every patch, in every layer and at every timestep. The waste is largest in pixel-space models, with no autoencoder to absorb low-level redundancy first. Probing a pretrained pixel text-to-image transformer, we find its middle-block tokens redundant wherever the image is flat. The redundancy occupies connected, content-shaped regions, and exploiting it requires tokens with the same geometry. Cutting a Hilbert ordering of the patches provides them. Consecutive positions are always image neighbours, so any contiguous run is a connected region whose size and shape follow the content, and grouping in two dimensions becomes a cut in one. Existing reductions each lose part of this. Similarity merging scatters its groups, latent bottlenecks discard position, and skipping deletes what it should summarize. We cut where the model's features change most and pool each run into one region token. Our Region Token Interface (\method{}) adapts a diffusion model to these tokens, with the region count drawn at random during fine-tuning so one checkpoint serves every budget. \method{} leads prior reduction methods at matched budgets, matches dense quality at $2.0\times$ the speed, and stays close at $2.6\times$. The code and models are open-sourced at https://eduardzamfir.github.io/rti

Related papers

• S$^2$Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models [5.680072178589744]
  Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens.<n>We propose S$2$Prune, a training-free pruning method that preserves spatial coverage while adapting token density to local image structure.
  arXiv  Detail & Related papers  (2026-09-01T13:26:57Z)
• MOSAIK: Multi-Patch Content-Aware Spatial Allocation of Image Tokens for Efficient Generation [26.941280093248263]
  MOSAIK is a damage-guided framework that varies patch size across regions and denoising steps.<n>Our approach delivers highly competitive performance at moderate budgets.
  arXiv  Detail & Related papers  (2026-08-05T22:52:41Z)
• ZipSplat: Fewer Gaussians, Better Splats [89.08529100444004]
  We propose ZipSplat, a token-based feed-forward model that decouples Gaussian placement from the pixel grid.<n>Cross- and self-attention refine these tokens, and a single trained model spans the quality-efficiency curve without retraining.<n>ZipSplat operates without ground-truth poses or intrinsics, yet sets a new state of the art on DL3DV and RealEstate10K with $sim6times$ fewer Gaussians than pixel-aligned methods.
  arXiv  Detail & Related papers  (2026-06-03T17:04:30Z)
• Everything at Every Scale: Scale-Invariant Diffusion with Continuous Super-Resolution [27.232771005158146]
  Creating images from noise is image generation; reconstructing fine details from coarse inputs is super-resolution.<n>We introduce $textbfSKILD$, a model that unifies generation and continuous super-resolution within a single unconditional framework.<n> Empirically, SKILD reaches FID $2.65$ and Inception Score $9.63$ on unconditional CIFAR-10, performs $2times$--$8times$ super-resolution on ImageNet from a single unconditional checkpoint.
  arXiv  Detail & Related papers  (2026-05-25T17:01:21Z)
• Efficient Scale-Invariant Generator with Column-Row Entangled Pixel Synthesis [3.222802562733787]
  We propose a new generative model that is both efficient and scale-equivariant without using any spatial convolutions or coarse-to-fine design. Experiments on various datasets, including FFHQ, LSUN-Church, MetFaces, and Flickr-Scenery, confirm CREPS' ability to synthesize scale-consistent and alias-free images.
  arXiv  Detail & Related papers  (2023-03-24T17:12:38Z)
• PATS: Patch Area Transportation with Subdivision for Local Feature Matching [78.67559513308787]
  Local feature matching aims at establishing sparse correspondences between a pair of images. We propose Patch Area Transportation with Subdivision (PATS) to tackle this issue. PATS improves both matching accuracy and coverage, and shows superior performance in downstream tasks.
  arXiv  Detail & Related papers  (2023-03-14T08:28:36Z)
• Improved Image Classification with Token Fusion [9.413225906164838]
  We propose a method using the fusion of CNN and transformer structure to improve image classification performance. In an experiment using ImageNet 1k, the proposed method shows the best classification performance.
  arXiv  Detail & Related papers  (2022-08-19T07:02:50Z)
• SWAT: Spatial Structure Within and Among Tokens [53.525469741515884]
  We argue that models can have significant gains when spatial structure is preserved during tokenization. We propose two key contributions: (1) Structure-aware Tokenization and, (2) Structure-aware Mixing.
  arXiv  Detail & Related papers  (2021-11-26T18:59:38Z)
• Large-scale image segmentation based on distributed clustering algorithms [70.8481702473572]
  Many approaches to 3D image segmentation are based on hierarchical clustering of supervoxels into image regions. Here we describe a distributed algorithm capable of handling a tremendous number of supervoxels. We demonstrate the algorithm by clustering an affinity graph with over 1.5 trillion edges between 135 billion supervoxels derived from a 3D electron microscopic brain image.
  arXiv  Detail & Related papers  (2021-06-21T01:11:49Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.