Dynamic Mixture-of-Experts for Visual Autoregressive Model

• URL: http://arxiv.org/abs/2510.08629v1
• Date: Wed, 08 Oct 2025 12:57:21 GMT
• Title: Dynamic Mixture-of-Experts for Visual Autoregressive Model
• Authors: Jort Vincenti, Metod Jazbec, Guoxuan Xia,
• Abstract summary: We introduce a dynamic Mixture-of-Experts router integrated into Visual Autoregressive Models.<n>We achieve 20% fewer FLOPs, 11% faster inference and match the image quality achieved by the dense baseline.
• Score: 6.51882364384472
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Visual Autoregressive Models (VAR) offer efficient and high-quality image generation but suffer from computational redundancy due to repeated Transformer calls at increasing resolutions. We introduce a dynamic Mixture-of-Experts router integrated into VAR. The new architecture allows to trade compute for quality through scale-aware thresholding. This thresholding strategy balances expert selection based on token complexity and resolution, without requiring additional training. As a result, we achieve 20% fewer FLOPs, 11% faster inference and match the image quality achieved by the dense baseline.

Related papers

• Deep Lightweight Unrolled Network for High Dynamic Range Modulo Imaging [19.49437461280304]
  Modulo-Imaging (MI) offers a promising alternative for expanding the dynamic dynamic range images by resetting the signal intensity when it reaches the intensity level.<n>We introduce the Scaling Equi term that facilitates self-tuning, thereby enabling the model to adapt to new images outside the original distribution.
  arXiv  Detail & Related papers  (2026-01-18T18:22:38Z)
• SFTok: Bridging the Performance Gap in Discrete Tokenizers [72.9996757048065]
  We propose textbfSFTok, a discrete tokenizer that incorporates a multi-step iterative mechanism for precise reconstruction.<n>At a high compression rate of only 64 tokens per image, SFTok achieves state-of-the-art reconstruction quality on ImageNet.
  arXiv  Detail & Related papers  (2025-12-18T18:59:04Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• PocketSR: The Super-Resolution Expert in Your Pocket Mobiles [69.26751136689533]
  Real-world image super-resolution (RealSR) aims to enhance the visual quality of in-the-wild images, such as those captured by mobile phones.<n>Existing methods leveraging large generative models demonstrate impressive results, but the high computational cost and latency make them impractical for edge deployment.<n>We introduce PocketSR, an ultra-lightweight, single-step model that brings generative modeling capabilities to RealSR while maintaining high fidelity.
  arXiv  Detail & Related papers  (2025-10-03T13:56:18Z)
• LoC-LIC: Low Complexity Learned Image Coding Using Hierarchical Feature Transforms [16.428925911432344]
  We propose an innovative approach that employs hierarchical feature extraction transforms to significantly reduce complexity.<n>Our novel architecture achieves this by using fewer channels for high spatial resolution inputs/feature maps.<n>As a result, the reduced complexity model can open the way for learned image compression models to operate efficiently across various devices.
  arXiv  Detail & Related papers  (2025-04-30T16:30:06Z)
• DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs [124.52164183968145]
  We present DyMU, an efficient, training-free framework that reduces the computational burden of vision-language models (VLMs)<n>Our approach comprises two key components. First, Dynamic Token Merging (DToMe) reduces the number of visual token embeddings by merging similar tokens based on image complexity.<n>Second, Virtual Token Unmerging (VTU) simulates the expected token sequence for large language models (LLMs) by efficiently reconstructing the attention dynamics of a full sequence.
  arXiv  Detail & Related papers  (2025-04-23T18:38:18Z)
• FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction [91.09318592542509]
  This work challenges the residual prediction paradigm in visual autoregressive modeling.<n>It presents a new Flexible Visual AutoRegressive image generation paradigm.<n>This simple, intuitive approach swiftly learns visual distributions and makes the generation process more flexible and adaptable.
  arXiv  Detail & Related papers  (2025-02-27T17:39:17Z)
• Exploring Linear Attention Alternative for Single Image Super-Resolution [28.267177967085143]
  Deep learning-based single-image super-resolution (SISR) technology focuses on enhancing low-resolution (LR) images into high-resolution (HR) ones.<n>We present a novel approach that combines the Receptance Weighted Key Value (RWKV) architecture with feature extraction techniques.<n>Under the 4x Super-Resolution tasks, compared to the MambaIR model, we achieved an average improvement of 0.26% in PSNR and 0.16% in SSIM.
  arXiv  Detail & Related papers  (2025-02-01T11:39:02Z)
• M-VAR: Decoupled Scale-wise Autoregressive Modeling for High-Quality Image Generation [39.97174784206976]
  We show that this scale-wise autoregressive framework can be effectively decoupled into textitintra-scale modeling We apply linear-complexity mechanisms like Mamba to substantially reduce computational overhead. Experiments demonstrate that our method outperforms existing models in both image quality and generation speed.
  arXiv  Detail & Related papers  (2024-11-15T18:54:42Z)
• Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration [100.54419875604721]
  All-in-one image restoration tackles different types of degradations with a unified model instead of having task-specific, non-generic models for each degradation. We propose DyNet, a dynamic family of networks designed in an encoder-decoder style for all-in-one image restoration tasks. Our DyNet can seamlessly switch between its bulkier and lightweight variants, thereby offering flexibility for efficient model deployment.
  arXiv  Detail & Related papers  (2024-04-02T17:58:49Z)
• An Efficient Implicit Neural Representation Image Codec Based on Mixed Autoregressive Model for Low-Complexity Decoding [43.43996899487615]
  Implicit Neural Representation (INR) for image compression is an emerging technology that offers two key benefits compared to cutting-edge autoencoder models. We introduce a new Mixed AutoRegressive Model (MARM) to significantly reduce the decoding time for the current INR. MARM includes our proposed AutoRegressive Upsampler (ARU) blocks, which are highly efficient, and ARM from previous work to balance decoding time and reconstruction quality.
  arXiv  Detail & Related papers  (2024-01-23T09:37:58Z)
• Transforming Image Super-Resolution: A ConvFormer-based Efficient Approach [58.57026686186709]
  We introduce the Convolutional Transformer layer (ConvFormer) and propose a ConvFormer-based Super-Resolution network (CFSR) CFSR inherits the advantages of both convolution-based and transformer-based approaches. Experiments demonstrate that CFSR strikes an optimal balance between computational cost and performance.
  arXiv  Detail & Related papers  (2024-01-11T03:08:00Z)
• Efficient Model Agnostic Approach for Implicit Neural Representation Based Arbitrary-Scale Image Super-Resolution [5.704360536038803]
  Single image super-resolution (SISR) has experienced significant advancements, primarily driven by deep convolutional networks. Traditional networks are limited to upscaling images to a fixed scale, leading to the utilization of implicit neural functions for generating arbitrarily scaled images. We introduce a novel and efficient framework, the Mixture of Experts Implicit Super-Resolution (MoEISR), which enables super-resolution at arbitrary scales.
  arXiv  Detail & Related papers  (2023-11-20T05:34:36Z)
• Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling [98.65190562585461]
  Diffusion models excel at generating photo-realistic images but come with significant computational costs. Current options like U-Net and Vision Transformer often rely on resource-intensive deep networks. This study introduces LEGO bricks, which seamlessly integrate Local-feature Enrichment and Global-content Orchestration.
  arXiv  Detail & Related papers  (2023-10-10T07:52:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.