ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction

• URL: http://arxiv.org/abs/2603.26258v1
• Date: Fri, 27 Mar 2026 10:26:54 GMT
• Title: ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction
• Abstract summary: ARTA is a mixed-resolution vision transformer for efficient dense feature extraction.<n>It starts with low-resolution (coarse) tokens and uses a lightweight allocator to predict which regions require more fine tokens.<n>Experiments demonstrate that ARTA achieves state-of-the-art results on ADE20K and COCO-Stuff with substantially fewer FLOPs.
• Score: 17.589359317827984
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present ARTA, a mixed-resolution coarse-to-fine vision transformer for efficient dense feature extraction. Unlike models that begin with dense high-resolution (fine) tokens, ARTA starts with low-resolution (coarse) tokens and uses a lightweight allocator to predict which regions require more fine tokens. The allocator iteratively predicts a semantic (class) boundary score and allocates additional tokens to patches above a low threshold, concentrating token density near boundaries while maintaining high sensitivity to weak boundary evidence. This targeted allocation encourages tokens to represent a single semantic class rather than a mixture of classes. Mixed-resolution attention enables interaction between coarse and fine tokens, focusing computation on semantically complex areas while avoiding redundant processing in homogeneous regions. Experiments demonstrate that ARTA achieves state-of-the-art results on ADE20K and COCO-Stuff with substantially fewer FLOPs, and delivers competitive performance on Cityscapes at markedly lower compute. For example, ARTA-Base attains 54.6 mIoU on ADE20K in the ~100M-parameter class while using fewer FLOPs and less memory than comparable backbones.

Related papers

• MOSAIK: Multi-Patch Content-Aware Spatial Allocation of Image Tokens for Efficient Generation [26.941280093248263]
  MOSAIK is a damage-guided framework that varies patch size across regions and denoising steps.<n>Our approach delivers highly competitive performance at moderate budgets.
  arXiv  Detail & Related papers  (2026-08-05T22:52:41Z)
• BATS: Resource-Efficient Volumetric Segmentation with Boundary-Aware Mixed-Resolution Tokens [14.29921504001782]
  We present BATS, a 3D medical image segmentation architecture that concentrates fine-resolution processing near predicted class boundaries.<n>A dense boundary predictor identifies where additional resolution is needed, while a fine-first context constructs an input-dependent mixed-resolution hierarchy.<n>Inference is up to 30% faster on KiTS and LiTS, which retain fewer tokens, but slower on the more token-dense BraTS.
  arXiv  Detail & Related papers  (2026-07-29T12:21:40Z)
• One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs [18.48496973561215]
  Large Vision-Language Models (LVLMs) have achieved remarkable success across diverse multimodal tasks, yet their practical deployment remains constrained by the computational burden arising from lengthy visual tokens.<n>We propose Adaptive Layer-wise Visual Token Selection (ALVTS), a novel framework that breaks away from the conventional static token pruning paradigm.<n>With an 89% token compression ratio, ALVTS retains 96.7% of the original model's accuracy, achieving a superior efficiency-accuracy trade-off for LVLM inference.
  arXiv  Detail & Related papers  (2026-06-12T08:58:58Z)
• MARCO: Navigating the Unseen Space of Semantic Correspondence [28.55177516749558]
  MARCO is a unified model for generalizable correspondence driven by a novel training framework.<n>Our approach transforms a handful of keypoints into dense, semantically coherent correspondences.<n>MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL.
  arXiv  Detail & Related papers  (2026-04-20T13:44:46Z)
• Latent-Condensed Transformer for Efficient Long Context Modeling [60.72493959155964]
  We propose Latent-Condensed Attention (LCA) for large language models.<n>LCA condenses context within MLA's latent space, where representation is disentangled into semantic latent vectors and positional keys.<n>LCA achieves up to 2.5$times$ prefilling speedup and 90% KV cache reduction at 128K context.
  arXiv  Detail & Related papers  (2026-04-14T08:40:31Z)
• Multi-Scale Local Speculative Decoding for Image Generation [10.239314110594249]
  We introduce Multi-Scale Local Speculative Decoding (MuLo-SD)<n>MuLo-SD combines multi-resolution drafting with spatially informed verification to accelerate AR image generation.<n>We demonstrate that MuLo-SD achieves substantial speedups up to $mathbf1.7times$.
  arXiv  Detail & Related papers  (2026-01-08T17:39:35Z)
• Saccadic Vision for Fine-Grained Visual Classification [10.681604440788854]
  Fine-grained visual classification (FGVC) requires distinguishing between visually similar categories through subtle, localized features.<n>Existing part-based methods rely on complex localization networks that learn mappings from pixel to sample space.<n>We propose a two-stage process that first extracts peripheral features and generates a sample map.<n>We employ contextualized selective attention to weigh the impact of each fixation patch before fusing peripheral and focus representations.
  arXiv  Detail & Related papers  (2025-09-19T07:03:37Z)
• Semantic Concentration for Self-Supervised Dense Representations Learning [103.10708947415092]
  Image-level self-supervised learning (SSL) has made significant progress, yet learning dense representations for patches remains challenging.<n>This work reveals that image-level SSL avoids over-dispersion by involving implicit semantic concentration.
  arXiv  Detail & Related papers  (2025-09-11T13:12:10Z)
• Exploiting Discriminative Codebook Prior for Autoregressive Image Generation [54.14166700058777]
  token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm.<n>While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited.<n>Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook.<n>We propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means
  arXiv  Detail & Related papers  (2025-08-14T15:00:00Z)
• VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models [35.38573641029626]
  We introduce the novel task of Extreme Short Token Reduction, which aims to represent entire videos using a minimal set of discrete tokens.<n>On the Extreme Short Token Reduction task, our VQToken compresses sequences to just 0.07 percent of their original length while incurring only a 0.66 percent drop in accuracy on the NextQA-MC benchmark.
  arXiv  Detail & Related papers  (2025-03-21T09:46:31Z)
• MST: Adaptive Multi-Scale Tokens Guided Interactive Segmentation [8.46894039954642]
  We propose a novel multi-scale token adaptation algorithm for interactive segmentation. By performing top-k operations across multi-scale tokens, the computational complexity is greatly simplified. We also propose a token learning algorithm based on contrastive loss.
  arXiv  Detail & Related papers  (2024-01-09T07:59:42Z)
• AiluRus: A Scalable ViT Framework for Dense Prediction [95.1313839257891]
  Vision transformers (ViTs) have emerged as a prevalent architecture for vision tasks owing to their impressive performance. We propose to apply adaptive resolution for different regions in the image according to their importance. We evaluate our proposed method on three different datasets and observe promising performance.
  arXiv  Detail & Related papers  (2023-11-02T12:48:43Z)
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE [51.09507030387935]
  Wasserstein autoencoder (WAE) shows that matching two distributions is equivalent to minimizing a simple autoencoder (AE) loss under the constraint that the latent space of this AE matches a pre-specified prior distribution. We propose to use the contrastive learning framework that has been shown to be effective for self-supervised representation learning, as a means to resolve this problem. We show that using the contrastive learning framework to optimize the WAE loss achieves faster convergence and more stable optimization compared with existing popular algorithms for WAE.
  arXiv  Detail & Related papers  (2021-10-19T22:55:47Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.