WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference

• URL: http://arxiv.org/abs/2604.17701v1
• Date: Mon, 20 Apr 2026 01:29:56 GMT
• Title: WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference
• Abstract summary: WISV (Wireless-Informed Semantic Verification) is a novel distributed speculative decoding framework.<n>We show that WISV achieves up to a 60.8% increase in accepted length, a 37.3% reduction in interaction rounds, and a 31.4% improvement in end-to-end latency.<n>We validate WISV on a hardware testbed comprising an NVIDIA Jetson AGX Orin and an A40-equipped server.
• Score: 56.297697169678095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While distributed device-edge speculative decoding enhances resource utilization across heterogeneous nodes, its performance is often bottlenecked by conventional token-level verification strategies. Such rigid alignment leads to excessive rejections, significantly diminishing the accepted sequence length and increasing interaction rounds under fluctuating wireless conditions. In this paper, we propose WISV (Wireless-Informed Semantic Verification), a novel distributed speculative decoding framework that goes beyond strict token-level matching via a channel-aware semantic acceptance policy. WISV integrates a lightweight decision head into the edge-side target LLM to dynamically evaluate speculative tokens by synthesizing high-dimensional hidden representations with instantaneous channel state information (CSI). To optimize the trade-off between verification fidelity and communication overhead, we further design two tailored communication protocols: full-hidden upload and mismatch-first selective-hidden upload. Extensive simulations using a 1B drafter and an 8B target model demonstrate that WISV achieves up to a 60.8% increase in accepted length, a 37.3% reduction in interaction rounds, and a 31.4% improvement in end-to-end latency compared to vanilla speculative decoding across tested settings, while maintaining a negligible task accuracy drop (<1%). Finally, we validate WISV on a hardware testbed comprising an NVIDIA Jetson AGX Orin and an A40-equipped server, confirming its real-world efficacy in accelerating edge-deployed LLM inference.

Related papers

• Task-Oriented Candidate-Latent Feedback for Coarse-to-Fine Sensing in Distributed OFDM-ISAC Networks [6.604029885021074]
  Future integrated sensing and communication (ISAC) architectures separate the sensing entity (SE) that acquires measurements from the sensing function (SF) that performs inference.<n>We propose a learning-based coarse-to-fine sensing pipeline with candidate-latent feedback for single-target estimation.
  arXiv  Detail & Related papers  (2026-08-04T08:26:06Z)
• BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis [2.8218290601001854]
  BTI-Net establishes bidirectional communication at every decoder level through two parallel pathways via Task Interaction Modules.<n>Uncertainty Proxy Attention gates each TIM output per instance and per level using three signals that capture cross-task alignment, scene complexity, and prediction confidence.<n>Experiments on three medical benchmarks spanning ultrasound, dermoscopy, and brain MRI demonstrate consistent improvements in segmentation IoU and classification accuracy over both encoder-sharing and decoder-interaction baselines.
  arXiv  Detail & Related papers  (2026-06-27T22:37:26Z)
• TRADE: Transducer-Augmented Decoder for Speech LLM [52.69726809996728]
  Speech Large Language Models (Speech LLMs) lack a principled mechanism for streaming inference.<n>We propose TRADE TRansducer-Augmented DEcoder, which augments a multimodal LLM with a transducer branch that shares the audio encoder.<n>Three design choices make the system accurate, streamable, and long-form capable.
  arXiv  Detail & Related papers  (2026-06-07T07:15:34Z)
• Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment [60.442064966340524]
  We present a prototype study on NVIDIA Jetson Orin Nano devices connected over WiFi.<n>Key finding is that the dominant bottleneck is not just network bandwidth but also the CPU- GPU staging during communication.<n>Experiments show that this strategy reduces latency by 65%-77% and energy consumption by 34%-52% relative to full-tensor exchange.
  arXiv  Detail & Related papers  (2026-05-25T10:39:28Z)
• Task-Oriented Communication for Human Action Understanding via Edge-Cloud Co-Inference [19.901979392669443]
  Smart sensing has created a growing demand for the accurate understanding of human action at the network edge.<n>Traditional approaches require massive video data to be transmitted from resource-constrained edge devices to powerful cloud servers.<n>We propose a task-oriented communication framework for human action understanding (TOAU) through edge-cloud collaboration.
  arXiv  Detail & Related papers  (2026-05-08T07:08:28Z)
• Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding [58.92526489742584]
  We propose provably lossless.<n> verification method that significantly boosts the expected number of accepted tokens.<n>We show that HSD yields consistent improvements in acceptance rates across diverse model families and benchmarks.
  arXiv  Detail & Related papers  (2026-01-09T11:10:29Z)
• ZK-SenseLM: Verifiable Large-Model Wireless Sensing with Selective Abstention and Zero-Knowledge Attestation [0.0]
  ZK-SenseLM is a secure and auditable wireless sensing framework.<n>It pairs a large-model encoder for Wi-Fi channel state information with a policy-grounded decision layer and zero-knowledge proofs of inference.
  arXiv  Detail & Related papers  (2025-10-29T16:43:07Z)
• CoSense-LLM: Semantics at the Edge with Cost- and Uncertainty-Aware Cloud-Edge Cooperation [0.0]
  CoSense-LLM is an edge-first framework that turns continuous multimodal sensor streams into compact semantic tokens.<n>System works with modern serving optimizations, including paged or streaming KV caches, Flash style kernels, speculative decoding, and quantized LoRA adapters.
  arXiv  Detail & Related papers  (2025-10-22T15:16:56Z)
• Adaptive Learning for IRS-Assisted Wireless Networks: Securing Opportunistic Communications Against Byzantine Eavesdroppers [7.256056777973974]
  We propose a joint learning framework for Byzantine-resilient spectrum sensing and secure intelligent reflecting surface (IRS)<n>We develop an augmented-Lagrangian alternating algorithm with projected updates and provide provable sublinear convergence, with accelerated rates under mild local curvature.<n> Simulations across diverse network conditions show higher detection probability at fixed false-alarm rate under adversarial attacks, large reductions in sum MSE for honest users, strong suppression of eavesdropper signal power, and fast convergence.
  arXiv  Detail & Related papers  (2025-08-11T17:28:25Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [54.53508601749513]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework.<n>To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features.<n>Results show that TOFC achieves up to 52% reduction in data transmission overhead and 63% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• Global Context Aggregation Network for Lightweight Saliency Detection of Surface Defects [70.48554424894728]
  We develop a Global Context Aggregation Network (GCANet) for lightweight saliency detection of surface defects on the encoder-decoder structure. First, we introduce a novel transformer encoder on the top layer of the lightweight backbone, which captures global context information through a novel Depth-wise Self-Attention (DSA) module. The experimental results on three public defect datasets demonstrate that the proposed network achieves a better trade-off between accuracy and running efficiency compared with other 17 state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-09-22T06:19:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.