SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Models

• URL: http://arxiv.org/abs/2608.15075v1
• Date: Sat, 15 Aug 2026 06:50:11 GMT
• Title: SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Models
• Abstract summary: RS-LVLMs have advanced multimodal understanding of Earth observation imagery, yet their performance is constrained by high-resolution processing.<n>Existing token pruning methods rely on scale-agnostic resolution policies and isolated importance cues.<n>We present Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning (SA-GEM), a plug-and-play framework that unifies task-adaptive token granularity allocation with holistic geospatial token importance modulation.
• Score: 52.99440326540357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RS-LVLMs have advanced multimodal understanding of Earth observation imagery, yet their performance is fundamentally constrained by high-resolution processing, as visual token counts grow quadratically with linear input resolution while important visual evidence is inherently sparse and increasingly diluted across the expanded sequence. Existing token pruning methods largely rely on scale-agnostic resolution policies and isolated importance cues, limiting task-aligned granularity adaptation and holistic evidence preservation. To address this, we present Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning (SA-GEM), a plug-and-play framework that unifies task-adaptive token granularity allocation with holistic geospatial token importance modulation. Specifically, a lightweight router selects the resolution based on query-dependent token granularity, while a token importance modulator jointly models task relevance, spatial structure, and local redundancy to preserve holistic geospatial evidence. We show that higher resolution is not universally beneficial and, once sufficient granularity is reached, token quality matters more than token quantity. Experiments across various benchmarks demonstrate that SA-GEM achieves consistent gains in both accuracy and efficiency over existing pruning methods. On XLRS-Bench, it surpasses GeoLLaVA-8K by 2.3% in accuracy with a 2.4 times total inference speedup.

Related papers

• Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models [12.459927405623624]
  Divide-and-Conquer Inference (DCI) is a novel test-time scaling strategy for visual recognition with MLLMs.<n>DCI decomposes complex global classification tasks into simpler, localized subproblems and employs a dynamic pruning mechanism to compress the search space.<n>As a model-agnostic, plug-and-play paradigm, DCI offers an efficient approach for scaling the inferential precision of MLLMs in large-scale scenarios.
  arXiv  Detail & Related papers  (2026-05-24T01:07:05Z)
• HM-Bench: A Comprehensive Benchmark for Multimodal Large Language Models in Hyperspectral Remote Sensing [22.804236694410367]
  multimodal large language models (MLLMs) have made significant strides in natural image understanding, but their ability to perceive and reason over hyperspectral image (HSI) remains underexplored.<n>We introduce Hyperspectral Multimodal Benchmark (HM-Bench), the first benchmark designed specifically to evaluate MLLMs in HSI understanding.<n>We curate a large-scale dataset of 19,337 question-answer pairs across 13 task categories, ranging from basic perception to spectral reasoning.
  arXiv  Detail & Related papers  (2026-04-10T02:47:32Z)
• Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models [41.32939197803798]
  Q-Zoom is a query-aware adaptive high-resolution perception framework.<n>A lightweight Gating Network safely bypasses high-resolution processing when coarse global features suffice.<n>A Self-Distilled Region Network precisely localizes the task-relevant Region-of-Interest from intermediate feature spaces.
  arXiv  Detail & Related papers  (2026-04-08T10:12:30Z)
• GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagery [69.05066425853326]
  "thinking-with-images" paradigm enables multimodal large language models (MLLMs) to actively explore visual scenes via zoom-in tools.<n>This is essential for ultra-high-resolution (UHR) remote sensing VQA, where task-relevant cues are sparse and tiny.<n>We propose GeoEyes, a training framework consisting of (1) a cold-start SFT dataset, UHR Chain-of-Zoom (UHR-CoZ), which covers diverse zooming regimes, and (2) an agentic reinforcement learning method, AdaZoom-GRPO, that explicitly rewards evidence gain and answer improvement during zoom
  arXiv  Detail & Related papers  (2026-02-15T15:50:55Z)
• GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing [50.961694646995376]
  We propose a parameter-efficient fine-tuning (PEFT) strategy called Guided Region-Aware Sparse Prompting (GRASP)<n>GRASP introduces spatially structured soft prompts associated with spatial blocks extracted from a frozen visual token grid.<n>Experiments on multiple RSVQA benchmarks show that GRASP achieves competitive performance compared to existing fine-tuning and prompt-based methods.
  arXiv  Detail & Related papers  (2026-01-23T10:12:59Z)
• Dynamic Granularity Matters: Rethinking Vision Transformers Beyond Fixed Patch Splitting [15.751224470424786]
  Vision Transformers (ViTs) have demonstrated strong capabilities in capturing global dependencies but often struggle to efficiently represent fine-grained local details.<n>We propose Granularity-driven Vision Transformer (Grc-ViT), a dynamic coarse-to-fine framework that adaptively adjusts visual granularity based on image complexity.<n>Two learnable parameters, and beta, are optimized end-to-end to balance global reasoning and local perception.
  arXiv  Detail & Related papers  (2025-11-24T11:55:22Z)
• HERO: Rethinking Visual Token Early Dropping in High-Resolution Large Vision-Language Models [60.028070589466445]
  We propose HERO, a framework that integrates content-adaptive token budget allocation with function-aware token selection.<n>This study provides both empirical insights and practical solutions toward efficient inference in HR-LVLMs.
  arXiv  Detail & Related papers  (2025-09-16T13:22:08Z)
• Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models [85.51753014478315]
  We introduce AdaptPrune, a novel plug-and-play training-free pruning method.<n>It builds on conventional attention-based pruning by integrating spatial distance and token similarity with an adaptive NMS approach.<n>Our approach ensures a comprehensive evaluation of token importance and substantially refines the pruning decisions.
  arXiv  Detail & Related papers  (2025-03-11T03:58:17Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Self-Guided Adaptation: Progressive Representation Alignment for Domain Adaptive Object Detection [86.69077525494106]
  Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the cross-domain robustness of object detection models. Existing UDA methods largely ignore the instantaneous data distribution during model learning, which could deteriorate the feature representation given large domain shift. We propose a Self-Guided Adaptation (SGA) model, target at aligning feature representation and transferring object detection models across domains.
  arXiv  Detail & Related papers  (2020-03-19T13:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.