Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge

• URL: http://arxiv.org/abs/2606.04581v2
• Date: Sun, 07 Jun 2026 06:19:01 GMT
• Title: Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge
• Abstract summary: We introduce Speculative inference as an efficient architecture to accelerate Large Language Models (LLMs)<n>In this work, we propose its distributed deployment to enable cooperative token generation in a multiuser edge system.<n>By developing decomposition methods, we reduce complex optimizations into tractable sub-problems.<n>Our analysis shows that the optimal bandwidth allocation compensates users with weaker computation-and-communication capabilities.
• Score: 47.08741228523925
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speculative inference (SPIN) was originally developed as an efficient architecture to accelerate Large Language Models (LLMs). In this work, we propose its distributed deployment to enable cooperative token generation in a multiuser edge system; its advantage is to effectively balance computational loads between resource-constrained devices and servers. The resulting architecture, termed Multi-access SPIN (Multi-SPIN), utilizes on-device small language models to generate and upload candidate token drafts, while an edge server operates the LLM to verify them in parallel batches. Given the severe heterogeneity in users' computation and communication capabilities, the draft length emerges as a critical control variable that influences node-level computation loads and multi-access latency, thereby governing the sum token goodput. Consequently, considering frequency-division multiple access, we investigate the problem of multi-access draft control, a joint optimization of draft-length control and bandwidth allocation to maximize sum token goodput. We examine two cases: (1) homogeneous draft lengths across users to facilitate server-side batching, and (2) heterogeneous draft lengths to introduce a new dimension for goodput enhancement. By developing decomposition methods, we reduce these complex optimizations into tractable sub-problems, which allow efficient draft control algorithms to be derived in closed form. Our analysis shows that the optimal bandwidth allocation compensates users with weaker computation-and-communication capabilities in the homogeneous case due to the batching synchronization requirements, whereas its heterogeneous-case counterpart rewards users with higher acceptance rates by relaxing such requirements. Experiments using Llama-2 and Qwen3.5 model pairs across diverse tasks demonstrate that Multi-SPIN improves goodput by up to 88% over heterogeneity-agnostic baselines.

Related papers

• BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks [12.929167421781074]
  Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks.<n>LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency.<n>We propose a hybrid autoregressive-speculative inference (BALANCE) framework for edge LLM inference.
  arXiv  Detail & Related papers  (2026-08-06T11:57:36Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• Learning Unified User Quantized Tokenizers for User Representation [32.942924291891636]
  U2QT (Unified User Quantized Tokenizers) is a novel framework that integrates cross-domain knowledge transfer with early fusion of heterogeneous domains.<n>Our framework employs a two-stage architecture: first, we use the Qwen3 Embedding model to derive a compact yet expressive feature representation.<n>Second, a multi-view RQ-VAE discretizes causal embeddings into compact tokens through shared and source-specific codebooks.
  arXiv  Detail & Related papers  (2025-08-01T08:35:32Z)
• Task-Oriented Multimodal Token Transmission in Resource-Constrained Multiuser Networks [19.42660454288912]
  We propose a task-oriented multimodal token transmission scheme for efficient multimodal information fusion and utilization.<n>To improve the efficiency of token transmission, we design a two-stage training algotithm, including cross-modal alignment and task-oriented fine-tuning.<n>We jointly optimize bandwidth, power allocation, and token length across users by using an alternating optimization method.
  arXiv  Detail & Related papers  (2025-05-06T14:17:05Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [54.53508601749513]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework.<n>To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features.<n>Results show that TOFC achieves up to 52% reduction in data transmission overhead and 63% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• Resource Management for Low-latency Cooperative Fine-tuning of Foundation Models at the Network Edge [35.40849522296486]
  Large-scale foundation models (FoMos) can perform human-like intelligence. FoMos need to be adapted to specialized downstream tasks through fine-tuning techniques. We advocate multi-device cooperation within the device-edge cooperative fine-tuning paradigm.
  arXiv  Detail & Related papers  (2024-07-13T12:47:14Z)
• Efficient Controllable Multi-Task Architectures [85.76598445904374]
  We propose a multi-task model consisting of a shared encoder and task-specific decoders where both encoder and decoder channel widths are slimmable. Our key idea is to control the task importance by varying the capacities of task-specific decoders, while controlling the total computational cost. This improves overall accuracy by allowing a stronger encoder for a given budget, increases control over computational cost, and delivers high-quality slimmed sub-architectures.
  arXiv  Detail & Related papers  (2023-08-22T19:09:56Z)
• Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI [108.08079323459822]
  This paper studies a new multi-intelligent edge artificial-latency (AI) system, which jointly exploits the AI model split inference and integrated sensing and communication (ISAC) We measure the inference accuracy by adopting an approximate but tractable metric, namely discriminant gain.
  arXiv  Detail & Related papers  (2022-07-03T06:57:07Z)
• Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channels [69.18343801164741]
  partitioned edge learning (PARTEL) implements parameter-server training, a well known distributed learning method, in wireless network. We consider the case of deep neural network (DNN) models which can be trained using PARTEL by introducing some auxiliary variables.
  arXiv  Detail & Related papers  (2020-10-08T15:27:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.