SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference

• URL: http://arxiv.org/abs/2607.00780v2
• Date: Thu, 02 Jul 2026 08:01:28 GMT
• Title: SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference
• Abstract summary: We present SpiralFovea, a parameter-free, input-adaptive tokeniser.<n>Around content-driven hotspot anchors, multi-scale spiral rings produce = 78 patches that replace the standard 196-patch ViT grid at the input stage.<n>Across four canonical fine-grained benchmarks, SpiralFovea yields +1.7-2.1 pp accuracy with a 60% reduction in input tokens, an 84% reduction in self-attention FLOPs at every transformer layer, and 18-29% throughput gains over the matched static tokenisation baseline.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most adaptive-inference techniques for foundation models change what the model does - early exit, MoE routing, KV-cache compression, dynamic attention sparsity. The input that hits the backbone, however, remains a fixed-grid tokenisation indifferent to image content. We argue that this is a missed lever. We present SpiralFovea, a parameter-free, input-adaptive tokeniser in which token identity, location, scale, and count are all functions of local visual entropy and selection completes before any backbone parameter is queried. Around content-driven hotspot anchors, multi-scale spiral rings produce <= 78 patches that replace the standard 196-patch ViT grid at the input stage. Across four canonical fine-grained benchmarks, SpiralFovea yields +1.7-2.1 pp accuracy with a 60% reduction in input tokens, an 84% reduction in self-attention FLOPs at every transformer layer, and 18-29% throughput gains over the matched static tokenisation baseline. A controlled ablation on CUB-200-2011 Genus across four backbones reveals a clean diagnostic: the gain magnitude tracks inversely with the strength of the backbone's whole-image positional prior, isolating self-supervised foundation models as the regime where input-adaptive tokenisation is most valuable.

Related papers

• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• PathSelect: Sequential Token Selection for Whole Slide Pathology [0.6362628914555645]
  Whole-Slide Images (WSIs) present a fundamental computational bottleneck for vision-language models (VLMs) due to extreme sequence lengths.<n>We reformulate WSI token pruning as a sequential selection process, enabling the model to autonomously learn an optimal routing strategy.
  arXiv  Detail & Related papers  (2026-07-26T12:36:16Z)
• Structural Kolmogorov-Arnold Convolutions: Learnable Function on the Values or the Filter Shape as Parameter-Efficient Alternative to Per-Edge Convolutional KANs [76.06235645266621]
  Convolutional Kolmogorov--Arnold Networks (KANs) replace the fixed weights of a convolutional kernel with learnable univariate functions.<n>We argue that the learnable functions are better placed in the emphstructure of the convolution than on each edge.
  arXiv  Detail & Related papers  (2026-06-23T10:00:26Z)
• HiRo: A Compact Four-Directional Hierarchical Reservoir Token-Mixer for Efficient Image Classification [0.19116784879310025]
  HiRo is a parameter-efficient image classification model that integrates shifted-window partitioning and hierarchical reservoir computing.<n>Images are divided into non-overlapping patches (treated as tokens), linearly projected, normalized, and enriched with 2D sinusoidal positional encodings, then processed within local windows.<n>Despite using under 1M trainable parameters, HiRo also achieves 99.46%, 85.57%, and 59.10% accuracy on MNIST, CIFAR-10, and CIFAR-100, respectively.
  arXiv  Detail & Related papers  (2026-06-13T06:44:06Z)
• Point-Wise Geometry-Aware Transformer for Partial-to-Full Point Cloud Registration in Computer-Assisted Surgery [5.569433775548014]
  We propose a learning-based point cloud registration framework with a coarse-to-fine architecture that combines convolution and transformer modules.<n>Experiments are conducted on four geometrically distinct bones, including the tibia, femur, pelvis, and thoracic cartilage.<n>The proposed method enables highly accurate 3D point cloud registration using partial observation.
  arXiv  Detail & Related papers  (2026-06-11T15:37:53Z)
• Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning [55.264863369127774]
  Current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.<n>We propose Hidden-Align, an auxiliary loss function that aligns the last-layer hidden states of correct rollouts at the anchor token during RL training, with zero overhead in both training and inference.<n>On eight mathematical reasoning benchmarks, Hidden-Align improves average pass@1 over the DAPO baseline by 3.8, 6.2, and 5.4 percentage points on Qwen3-1.7B, 4B, and 14B respectively, with consistent pass@k gains across all three scales.
  arXiv  Detail & Related papers  (2026-06-02T06:51:15Z)
• AdaMerge: Salience-Aware Adaptive Token Merging for Training-Free Acceleration of Vision Transformers [5.739405014622565]
  AdaMerge is a token-merging framework based on two complementary mechanisms.<n>It consistently outperforms ToMe, PiToMe, and DSM across all FLOPs-matched regimes.
  arXiv  Detail & Related papers  (2026-05-26T05:35:39Z)
• Faster or Stronger: Towards Flexible Visual Place Recognition via Weighted Aggregation and Token Pruning [27.4981354426677]
  Visual Place Recognition (VPR) aims to match a query image to reference images of the same place in a large-scale database.<n>We introduce WeiToP, a VPR-oriented token pruning framework that reduces feature extraction cost via self-distillation.<n>WeiToP enables plug-and-play token pruning at inference time, allowing flexible and on-demand control over the accuracy-efficiency trade-off without additional training.
  arXiv  Detail & Related papers  (2026-05-19T23:01:57Z)
• SEER: Spectral Entropy Encoding of Roles for Context-Aware Attention-Based Design Pattern Detection [0.0]
  This paper presents an upgraded version of our prior method Context Is All You Need for detecting Gang of Four (GoF) design patterns from source code.<n> SEER addresses these limitations with two principled additions: (i) a spectral-entropy role encoder that derives per-member role embeddings from the Laplacian spectrum of each class's interaction graph, and (ii) a time-weighted calling context that assigns empirically calibrated duration priors to method categories.<n>We evaluate SEER on PyDesignNet (1,832 files, 35,000 sequences, 23 GoF patterns) and observe consistent gains over our previous system
  arXiv  Detail & Related papers  (2026-01-19T19:13:40Z)
• DoPE: Denoising Rotary Position Embedding [60.779039511252584]
  Rotary Position Embedding (RoPE) in Transformer models has inherent limits that weaken length.<n>We reinterpret the attention map with positional encoding as a noisy feature map, and propose Denoising Positional extrapolation page (DoPE)<n>DoPE is a training-free method based on truncated matrix entropy to detect outlier frequency bands in the feature map.
  arXiv  Detail & Related papers  (2025-11-12T09:32:35Z)
• Sparse is Enough in Fine-tuning Pre-trained Large Language Models [98.46493578509039]
  We propose a gradient-based sparse fine-tuning algorithm, named Sparse Increment Fine-Tuning (SIFT) We validate its effectiveness on a range of tasks including the GLUE Benchmark and Instruction-tuning.
  arXiv  Detail & Related papers  (2023-12-19T06:06:30Z)
• Centroid-centered Modeling for Efficient Vision Transformer Pre-training [44.24223088955106]
  Masked Image Modeling (MIM) is a new self-supervised vision pre-training paradigm using a Vision Transformer (ViT) Our proposed centroid-based approach, CCViT, leverages k-means clustering to obtain centroids for image modeling without supervised training of the tokenizer model. Our approach achieves competitive results with recent baselines without external supervision and distillation training from other models.
  arXiv  Detail & Related papers  (2023-03-08T15:34:57Z)
• Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost [53.746169882193456]
  Recent works have proposed various sparse attention modules to overcome the quadratic cost of self-attention. We propose a model that resolves both problems by endowing each attention head with a mixed-membership Block Model. Our model outperforms previous efficient variants as well as the original Transformer with full attention.
  arXiv  Detail & Related papers  (2022-10-27T15:30:52Z)
• Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers [55.90468016961356]
  We propose an efficient token mixer that learns to mix in the Fourier domain. AFNO is based on a principled foundation of operator learning. It can handle a sequence size of 65k and outperforms other efficient self-attention mechanisms.
  arXiv  Detail & Related papers  (2021-11-24T05:44:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.