Depth Adaptive Efficient Visual Autoregressive Modeling

• URL: http://arxiv.org/abs/2604.17286v1
• Date: Sun, 19 Apr 2026 06:59:34 GMT
• Title: Depth Adaptive Efficient Visual Autoregressive Modeling
• Abstract summary: Visual Autoregressive ( VAR) modeling inefficiently applies a fixed computational depth to each position when generating high-resolution images.<n>We propose a paradigm shift from pruning entire tokens to adaptively allocating per-token computational depth.
• Score: 18.16894544431278
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Visual Autoregressive (VAR) modeling inefficiently applies a fixed computational depth to each position when generating high-resolution images. While existing methods accelerate inference by pruning tokens using frequency maps, their binary hard-pruning approach is fundamentally limited and fails to improve quality even with better frequency estimation. Observing that VAR models possess significant depth redundancy, we propose a paradigm shift from pruning entire tokens to adaptively allocating per-token computational depth. To this end, we introduce DepthVAR, a training-free framework that dynamically allocates computation. It integrates an adaptive depth scheduler, which assigns computational depth via a cyclic rotated schedule for balanced, non-static refinement, with a dynamic inference process that translates these depths into layer-major masks, selectively applies transformer blocks, and blends the resulting codes to ensure each token's influence is proportional to its processing depth. Extensive experiments show that DepthVAR achieves 2.3$\times$-3.1$\times$ acceleration with minimal quality loss, offering a competitive compute-performance trade-off compared to existing hard-pruning approaches. Code is available at https://github.com/STOVAGtz/DepthVAR

Related papers

• PARE: Pruning and Adaptive Routing for Efficient Video Generation [71.54959622788608]
  Video Diffusion Transformers (DiTs) generate high-quality videos but demand substantial compute due to wide blocks, deep architectures, and iterative sampling.<n>Recent methods reduce cost by compressing width, depth, or sampling steps, but typically commit to a fixed architecture that cannot adapt to individual inputs or denoising stages.<n>We propose PARE, which jointly compresses width and depth with structure-aware pruning and input-adaptive routing.
  arXiv  Detail & Related papers  (2026-05-26T17:43:25Z)
• TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Acceleration [64.32072516882947]
  Diffusion Policy excels in embodied control but suffers from high inference latency and computational cost.<n>We propose Temporal-aware Reinforcement-based Speculative Diffusion Policy (TS-DP)<n>TS-DP achieves up to 4.17 times faster inference with over 94% accepted drafts, reaching an inference frequency of 25 Hz.
  arXiv  Detail & Related papers  (2025-12-13T07:53:14Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging [45.39911367007956]
  Deep-unrolling and plug-and-play approaches have become the de-facto for single-pixel imaging (SPI) inverse problem.<n>In this paper, we address the challenge of integrating the strengths of both classes of solvers.
  arXiv  Detail & Related papers  (2025-05-29T07:16:57Z)
• High-Frequency Prior-Driven Adaptive Masking for Accelerating Image Super-Resolution [87.56382172827526]
  High-frequency regions are most critical for reconstruction.<n>We propose a training-free adaptive masking module for acceleration.<n>Our method reduces FLOPs by 24--43% for state-of-the-art models.
  arXiv  Detail & Related papers  (2025-05-11T13:18:03Z)
• PatchRefiner V2: Fast and Lightweight Real-Domain High-Resolution Metric Depth Estimation [38.71875790942604]
  PRV2 outperforms state-of-the-art depth estimation methods on UnrealStereo4K in both accuracy and speed.<n>It also shows improved depth boundary delineation on real-world datasets like CityScape, ScanNet++, and KITTI.
  arXiv  Detail & Related papers  (2025-01-02T07:41:27Z)
• Variational Bayes image restoration with compressive autoencoders [6.689746581015932]
  Regularization of inverse problems is paramount to importance in computational imaging.<n>In this work, we first propose to use variational autoencoders instead of state-of-the-art generative models.<n>As a second contribution, we introduce the Variational Bayes Latent Estimation (VBLE) algorithm, which performs latent estimation within variational inference.
  arXiv  Detail & Related papers  (2023-11-29T15:49:31Z)
• Dynamic Frame Interpolation in Wavelet Domain [57.25341639095404]
  Video frame is an important low-level computation vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts.
  arXiv  Detail & Related papers  (2023-09-07T06:41:15Z)
• FaDIV-Syn: Fast Depth-Independent View Synthesis [27.468361999226886]
  We introduce FaDIV-Syn, a fast depth-independent view synthesis method. Our multi-view approach addresses the problem that view synthesis methods are often limited by their depth estimation stage.
  arXiv  Detail & Related papers  (2021-06-24T16:14:01Z)
• Layer Pruning on Demand with Intermediate CTC [50.509073206630994]
  We present a training and pruning method for ASR based on the connectionist temporal classification (CTC) We show that a Transformer-CTC model can be pruned in various depth on demand, improving real-time factor from 0.005 to 0.002 on GPU.
  arXiv  Detail & Related papers  (2021-06-17T02:40:18Z)
• Faster Depth-Adaptive Transformers [71.20237659479703]
  Depth-adaptive neural networks can dynamically adjust depths according to the hardness of input words. Previous works generally build a halting unit to decide whether the computation should continue or stop at each layer. In this paper, we get rid of the halting unit and estimate the required depths in advance, which yields a faster depth-adaptive model.
  arXiv  Detail & Related papers  (2020-04-27T15:08:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.