Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference

• URL: http://arxiv.org/abs/2604.08971v1
• Date: Fri, 10 Apr 2026 05:26:17 GMT
• Title: Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
• Abstract summary: multimodal pipelines must remain accurate despite fluctuating power budgets and unpredictable sensor dropout.<n>SentryGate learns modality-conditioned importance scores during training via first-order saliency supervision.<n>SentryFuse reduces memory by 28.2% and lowers latency by up to $1.63times$ without further fine-tuning.
• Score: 9.096132303386264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Edge devices increasingly run multimodal sensing pipelines that must remain accurate despite fluctuating power budgets and unpredictable sensor dropout. Existing pruning methods fail under these conditions: they generally require fine-tuning after compression, consuming over $10\times$ the deployment energy, and they assign static importance scores that are blind to which sensors are present. We present the SentryFuse framework, which addresses both challenges jointly through two key components. First, SentryGate learns modality-conditioned importance scores during training via first-order saliency supervision and then prunes attention heads and feed-forward channels at deployment without fine-tuning. Second, SentryAttend replaces dense self-attention, a key bottleneck in contemporary multimodal architectures, with sparse grouped-query attention, yielding a net 15% reduction in GFLOPs across three different multimodal architectures. Across three applications and multimodal backbones, SentryGate achieves a 12.7% average accuracy improvement over the strongest pruning baseline, and upto to 18% under modality dropout conditions. Together, SentryFuse reduces memory by 28.2% and lowers latency by up to $1.63\times$ without further fine-tuning, establishing modality-aware zero-shot compression as a practical path to multimodal intelligence on heterogeneous edge hardware.

Related papers

• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• DREAM Technical Report [53.37525197047151]
  We present DREAM, an autonomous optimization control architecture that adds a perception-aware, orchestrable, and auditable policy layer atop existing pipelines without replacing them.<n> DREAM has two core components. First, a three-tier Intent Engine fuses on-device signals into structured L0/L1/L2 intent representations; its edge-cloud trigger chain reduces reporting volume to approximately 8.7%.<n>Second, a Meta Engine uses a MetaModel for layered M1-to-M2-to-M3 reasoning: intent summarization, strategy planning informed by Strategy Memory, and parameter translation. It dispatches the resulting parameters through a unified outlet with
  arXiv  Detail & Related papers  (2026-08-10T10:35:32Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• Device-Independent Quantum Secret Sharing Protocol Enhanced by Advantage Distillation [7.352414733286665]
  Device-independent quantum secret sharing (DI-QSS) provides high security by eliminating the need to trust devices.<n>This work extends advantage distillation from two-party quantum key distribution (QKD) to three-party DI-QSS.
  arXiv  Detail & Related papers  (2026-05-21T01:42:22Z)
• FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence [8.780507015964234]
  FusionSense is a fusion-aware intelligent sensing framework for energy-constrained autonomous edge systems.<n>On a dual-modality (RGB+Depth/LiDAR) setup with SynDrone, FusionSense sustains task quality at substantially higher data-reduction rates than uni-modal filters.
  arXiv  Detail & Related papers  (2026-05-19T21:59:32Z)
• Joint Architecture-Token-Bitwidth Multi-Axis Optimization of Vision Transformers for Semiconductor IC Packaging [32.78037242946664]
  Vision Transformers (ViTs) have achieved strong performance in visual recognition, yet their deployment in resource-constrained industrial environments remains limited.<n>We present one of the first holistic frameworks that jointly optimize three complementary axes: architecture, token, and bit-width.<n>Specifically, the framework identifies compact backbones via Neural Architecture Search (AutoFormer), reduces information processing via token merging (ToMe), and accelerates per-operation execution via fp16 mixed-precision inference.
  arXiv  Detail & Related papers  (2026-05-03T06:45:24Z)
• Phase-Scheduled Multi-Agent Systems for Token-Efficient Coordination [0.0]
  Phase-Scheduled Multi-Agent Systems (PSMAS) is a framework that reconceptualizes agent activation as continuous control over a shared attention space modeled on a circular manifold.<n>PSMAS achieves a mean token reduction of 27.3 percent (range 21.4-34.8 percent) while maintaining task performance within 2.1 percentage points of a fully activated baseline.
  arXiv  Detail & Related papers  (2026-04-19T12:03:54Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model [112.9420001646428]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost.<n>We present A1, a fully open-source and transparent VLA framework designed for low-cost, high- throughput inference.<n>A1 achieves state-of-the-art success rates while significantly reducing inference cost.
  arXiv  Detail & Related papers  (2026-04-07T10:18:40Z)
• PRISM: Performer RS-IMLE for Single-pass Multisensory Imitation Learning [51.24484551729328]
  We introduce PRISM, a single-pass policy based on a batch-global rejection-sampling variant of IMLE.<n> PRISM couples a temporal multisensory encoder with a linear-attention generator using a Performer architecture.<n>We demonstrate the efficacy of PRISM on a diverse real-world hardware suite, including loco-manipulation using a Unitree Go2 with a 7-DoF arm D1 and tabletop manipulation with a UR5 manipulator.
  arXiv  Detail & Related papers  (2026-02-02T17:57:37Z)
• Progressive Supernet Training for Efficient Visual Autoregressive Modeling [56.15415456746672]
  We propose a training strategy that breaks through the frontier of generation quality for both paradigms and the full network.<n>Experiments on ImageNet demonstrate that, compared to the pretrained VAR-d30, VARiant-d16 and VARiant-d8 achieve nearly equivalent quality.<n> VARiant-d2 achieves 3.5 times speedup and 80% memory reduction at moderate quality cost.
  arXiv  Detail & Related papers  (2025-11-20T16:59:24Z)
• Enabling Vibration-Based Gesture Recognition on Everyday Furniture via Energy-Efficient FPGA Implementation of 1D Convolutional Networks [11.481972015296812]
  This study proposes an energy-efficient solution deploying compact NNs on low-power Field-Programmable Gate Arrays (FPGAs)<n>We replace complex spectral preprocessing with raw waveform input, eliminating complex on-board preprocessing while reducing input size by 21x without sacrificing accuracy.<n>We design two lightweight architectures (1D-CNN and 1D-SepCNN) tailored for embedded FPGAs, reducing parameters from 369 million to as few as 216 while maintaining comparable accuracy.
  arXiv  Detail & Related papers  (2025-10-27T09:30:36Z)
• vAttention: Verified Sparse Attention [100.98210818821688]
  vAttention is a practical sparse attention mechanism with user-specified $(epsilon, delta)$ guarantees on approximation accuracy (thus, verified)<n>We show that vAttention significantly improves the quality of sparse attention across datasets.<n>It can be deployed in reasoning scenarios to achieve fast decoding without compromising model quality.
  arXiv  Detail & Related papers  (2025-10-07T08:46:08Z)
• FlashOmni: A Unified Sparse Attention Engine for Diffusion Transformers [7.026182341295719]
  Flash Omni is a unified sparse attention engine compatible with arbitrary DiT architectures.<n>It delivers near-linear, closely matching the sparsity ratio speedup in attention and GEMM-$Q$, and achieves 2.5$times$-3.8$times$ acceleration in GEMM-$O$.
  arXiv  Detail & Related papers  (2025-09-29T18:57:14Z)
• Performance Characterization of a Multi-Module Quantum Processor with Static Inter-Chip Couplers [63.42120407991982]
  Three-dimensional integration technologies such as flip-chip bonding are a key prerequisite to realize large-scale superconducting quantum processors.<n>We present a design for a multi-chip module comprising one carrier chip and four qubit modules.<n>Measuring two of the qubits, we analyze the readout performance, finding a mean three-level state-assignment error of $9 times 10-3$ in 200 ns.<n>We demonstrate a controlled-Z two-qubit gate in 100 ns with an error of $7 times 10-3$ extracted from interleaved randomized benchmarking.
  arXiv  Detail & Related papers  (2025-03-16T18:32:44Z)
• Duo Streamers: A Streaming Gesture Recognition Framework [9.128440633129186]
  Duo Streamers is a streaming gesture recognition framework for resource-constrained devices.<n>It achieves high accuracy and low latency through a three-stage sparse recognition mechanism, an RNN-lite model with an external hidden state, and specialized training and post-processing pipelines.
  arXiv  Detail & Related papers  (2025-02-17T20:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.