Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning

• URL: http://arxiv.org/abs/2606.08641v1
• Date: Sun, 07 Jun 2026 14:07:28 GMT
• Title: Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning
• Abstract summary: We reformulate token reduction in whole slide images as a trainable sparsification problem.<n>We show that our framework achieves 73.32% overall accuracy on SlideBench (TCGA)<n>This work provides a highly efficient paradigm for end to end gigapixel whole slide image reasoning.
• Score: 8.596353244934853
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The processing of gigapixel whole slide images within vision language models faces a major difficulty due to an excessive number of visual tokens. Existing solutions typically rely on spatial downsampling or heuristic pruning strategies that operate without training, and these methods often discard subtle but clinically meaningful patterns because pathological evidence is scattered irregularly across the tissue. To overcome this limitation, we reformulate token reduction in whole slide images as a trainable sparsification problem, allowing the model to learn an optimal selection strategy instead of following fixed heuristics. We propose a decoupled routing architecture. To enable gradient propagation through the nondifferentiable pruning operation during training, we introduce a component called SparseLearn. This component uses a variance-preserving noise gate that regulates the information flow of each patch via a differentiable Soft Top-K operator, together with a diagonal attention denoiser that recovers perturbed representations without leaking spatial information. At inference time, the SparseLearn module is entirely discarded, and the trained scorer applies a deterministic Hard Top-K operator to keep only the highest scoring 32 tokens, incurring no extra computation. By compressing the visual sequence down to a sparse set of just 32 tokens, which represents as little as 0.78% of the original length, our framework achieves 73.32% overall accuracy on SlideBench (TCGA), consistently surpassing sampling-based baselines and general-purpose vision language models. It also demonstrates strong zero shot generalization on SlideBench (BCNB) and WSI VQA*. By resolving the visual context bottleneck and preventing the dilution of sparse diagnostic evidence, this work provides a highly efficient paradigm for end to end gigapixel whole slide image reasoning.

Related papers

• PathSelect: Sequential Token Selection for Whole Slide Pathology [0.6362628914555645]
  Whole-Slide Images (WSIs) present a fundamental computational bottleneck for vision-language models (VLMs) due to extreme sequence lengths.<n>We reformulate WSI token pruning as a sequential selection process, enabling the model to autonomously learn an optimal routing strategy.
  arXiv  Detail & Related papers  (2026-07-26T12:36:16Z)
• TC-SSA: Token Compression via Semantic Slot Aggregation for Gigapixel Pathology Reasoning [10.675615956951468]
  We propose TC-SSA, a learnable token compression framework that aggregates patch features into a fixed number of semantic slots.<n>A gated routing module assigns patches to slots using sparse Top-2 routing, followed by weighted aggregation, enabling global slide coverage under a strict token budget.<n>On the SlideBench(TCGA), our model achieves 78.34% overall accuracy and 77.14% on the diagnosis subset, outperforming sampling-based baselines.
  arXiv  Detail & Related papers  (2026-03-01T15:07:57Z)
• Switchable Token-Specific Codebook Quantization For Face Image Compression [72.44596412563503]
  We propose a Switchable Token-Specific Codebook Quantization for face image compression.<n>By recording the codebook group to which each token belongs with a small number of bits, our method can reduce the loss incurred when decreasing the size of each codebook group.<n>Our method has demonstrated its effectiveness on face recognition datasets, achieving an average accuracy of 93.51% for reconstructed images at 0.05 bpp.
  arXiv  Detail & Related papers  (2025-10-27T02:56:17Z)
• Approximate Size Targets Are Sufficient for Accurate Semantic Segmentation [52.239136918460616]
  Extending binary class tags to approximate relative object-size distributions allows off-the-shelf architectures to solve the segmentation problem.<n>A straightforward zero-avoiding KL-divergence loss for average predictions produces segmentation accuracy comparable to the standard pixel-precise supervision.<n>Our ideas are validated on PASCAL VOC using our new human annotations of approximate object sizes.
  arXiv  Detail & Related papers  (2025-03-10T06:02:13Z)
• Cascaded Cross-Attention Networks for Data-Efficient Whole-Slide Image Classification Using Transformers [0.11219061154635457]
  Whole-Slide Imaging allows for the capturing and digitization of high-resolution images of histological specimen. transformer architecture has been proposed as a possible candidate for effectively leveraging the high-resolution information. We propose a novel cascaded cross-attention network (CCAN) based on the cross-attention mechanism that scales linearly with the number of extracted patches.
  arXiv  Detail & Related papers  (2023-05-11T16:42:24Z)
• DBAT: Dynamic Backward Attention Transformer for Material Segmentation with Cross-Resolution Patches [8.812837829361923]
  We propose the Dynamic Backward Attention Transformer (DBAT) to aggregate cross-resolution features. Experiments show that our DBAT achieves an accuracy of 86.85%, which is the best performance among state-of-the-art real-time models. We further align features to semantic labels, performing network dissection, to infer that the proposed model can extract material-related features better than other methods.
  arXiv  Detail & Related papers  (2023-05-06T03:47:20Z)
• Learning strides in convolutional neural networks [34.20666933112202]
  This work introduces DiffStride, the first downsampling layer with learnable strides. Experiments on audio and image classification show the generality and effectiveness of our solution.
  arXiv  Detail & Related papers  (2022-02-03T16:03:36Z)
• Mixed Supervision Learning for Whole Slide Image Classification [88.31842052998319]
  We propose a mixed supervision learning framework for super high-resolution images. During the patch training stage, this framework can make use of coarse image-level labels to refine self-supervised learning. A comprehensive strategy is proposed to suppress pixel-level false positives and false negatives.
  arXiv  Detail & Related papers  (2021-07-02T09:46:06Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Powers of layers for image-to-image translation [60.5529622990682]
  We propose a simple architecture to address unpaired image-to-image translation tasks. We start from an image autoencoder architecture with fixed weights. For each task we learn a residual block operating in the latent space, which is iteratively called until the target domain is reached.
  arXiv  Detail & Related papers  (2020-08-13T09:02:17Z)
• Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches [52.67723703088284]
  We propose a novel framework called multi-patch generative adversarial nets (MPGAN) MPGAN synthesises local patch features and labels unseen classes with a novel weighted voting strategy. MPGAN has significantly greater accuracy than state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-27T05:49:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.