EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation

• URL: http://arxiv.org/abs/2603.12267v1
• Date: Thu, 12 Mar 2026 17:59:59 GMT
• Title: EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation
• Abstract summary: EVATok is a framework to produce $textbfE$fficient $textbfV$ideo $textbfA$daptive $textbfTok$enizers.<n>We show that EVATok achieves superior reconstruction and state-of-the-art class-to-video generation on UCF-101.
• Score: 80.13014959623452
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive (AR) video generative models rely on video tokenizers that compress pixels into discrete token sequences. The length of these token sequences is crucial for balancing reconstruction quality against downstream generation computational cost. Traditional video tokenizers apply a uniform token assignment across temporal blocks of different videos, often wasting tokens on simple, static, or repetitive segments while underserving dynamic or complex ones. To address this inefficiency, we introduce $\textbf{EVATok}$, a framework to produce $\textbf{E}$fficient $\textbf{V}$ideo $\textbf{A}$daptive $\textbf{Tok}$enizers. Our framework estimates optimal token assignments for each video to achieve the best quality-cost trade-off, develops lightweight routers for fast prediction of these optimal assignments, and trains adaptive tokenizers that encode videos based on the assignments predicted by routers. We demonstrate that EVATok delivers substantial improvements in efficiency and overall quality for video reconstruction and downstream AR generation. Enhanced by our advanced training recipe that integrates video semantic encoders, EVATok achieves superior reconstruction and state-of-the-art class-to-video generation on UCF-101, with at least 24.4% savings in average token usage compared to the prior state-of-the-art LARP and our fixed-length baseline.

Related papers

• TrajTok: Learning Trajectory Tokens enables better Video Understanding [63.1260672430712]
  Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens.<n>We propose TrajTok, an end-to-end video tokenizer module that is fully integrated and co-trained with video models for a downstream objective.<n>We show that it can be seamlessly integrated as either a probing head for pretrained visual features (TrajAdapter) or an alignment connector in vision-language models (TrajVLM) with especially strong performance in long-video reasoning.
  arXiv  Detail & Related papers  (2026-02-26T09:15:34Z)
• CoPE-VideoLM: Codec Primitives For Efficient Video Language Models [56.76440182038839]
  Video Language Models (VideoLMs) empower AI systems to understand temporal dynamics in videos.<n>Current methods use sampling which can miss both macro-level events and micro-level details due to the sparse temporal coverage.<n>We propose to leverage video primitives which encode video redundancy and sparsity without requiring expensive full-image encoding for most frames.
  arXiv  Detail & Related papers  (2026-02-13T18:57:31Z)
• Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Context [8.458436768725212]
  Video autoencoders compress videos into compact latent representations for efficient reconstruction.<n>We propose Autoregressive Video Autoencoder (ARVAE), which compresses and reconstructs each frame conditioned on its predecessor in an autoregressive manner.<n>ARVAE achieves superior reconstruction quality with extremely lightweight models and small-scale training data.
  arXiv  Detail & Related papers  (2025-12-12T05:40:01Z)
• ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation [24.639936266140385]
  Existing static pruning methods degrade performance by permanently removing weights or tokens.<n>We propose Act VAR, a dynamic activation framework that introduces dual sparsity across model weights and token sequences.<n>Experiments on the ImageNet $256times 256$ benchmark demonstrate that Act VAR achieves up to $21.2%$ FLOPs reduction with minimal performance degradation.
  arXiv  Detail & Related papers  (2025-11-17T02:28:06Z)
• Learning Adaptive and Temporally Causal Video Tokenization in a 1D Latent Space [94.07013629356113]
  AdapTok is an adaptive temporal causal video tokenizer that can flexibly allocate tokens for different frames based on video content.<n>AdapTok consistently improves reconstruction quality and generation performance under different token budgets.
  arXiv  Detail & Related papers  (2025-05-22T17:59:02Z)
• STORM: Token-Efficient Long Video Understanding for Multimodal LLMs [116.4479155699528]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs.<n>We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• Efficient Long Video Tokenization via Coordinate-based Patch Reconstruction [93.69757398746017]
  CoordTok is a video tokenizer that learns a mapping from coordinate-based representations to the corresponding patches of input videos.<n>CoordTok encodes a video into factorized triplane representations and reconstructs patches that correspond to randomly sampled $(x,y,t)$ coordinates.
  arXiv  Detail & Related papers  (2024-11-22T06:50:44Z)
• LARP: Tokenizing Videos with a Learned Autoregressive Generative Prior [36.663855554010674]
  We present LARP, a novel video tokenizer designed to overcome limitations in current video tokenization methods for autoregressive (AR) generative models.<n>Unlike traditional patchwise tokenizers that directly encode local visual patches into discrete tokens, LARP introduces a holistic tokenization scheme.<n>It captures more global and semantic representations, rather than being limited to local patch-level information.
  arXiv  Detail & Related papers  (2024-10-28T17:57:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.