EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs

• URL: http://arxiv.org/abs/2511.01866v1
• Date: Tue, 21 Oct 2025 04:18:25 GMT
• Title: EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs
• Authors: Benjamin Kubwimana, Qijing Huang,
• Abstract summary: Large language models (LLMs) for reasoning tasks on edge GPU face critical challenges from strict latency constraints and limited computational resources.<n>To navigate these constraints, developers must balance reasoning versus non-reasoning architectures, selecting appropriate model sizes, allocating token budgets, and applying test-time scaling strategies.<n>We present EdgeReasoning, a comprehensive study characterizing the deployment of reasoning LLMs on edge GPUs.
• Score: 0.36050743818632486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Edge intelligence paradigm is increasingly demanded by the emerging autonomous systems, such as robotics. Beyond ensuring privacy-preserving operation and resilience in connectivity-limited environments, edge deployment offers significant energy and cost advantages over cloud-based solutions. However, deploying large language models (LLMs) for reasoning tasks on edge GPUs faces critical challenges from strict latency constraints and limited computational resources. To navigate these constraints, developers must balance multiple design factors - choosing reasoning versus non-reasoning architectures, selecting appropriate model sizes, allocating token budgets, and applying test-time scaling strategies - to meet target latency and optimize accuracy. Yet guidance on optimal combinations of these variables remains scarce. In this work, we present EdgeReasoning, a comprehensive study characterizing the deployment of reasoning LLMs on edge GPUs. We systematically quantify latency-accuracy tradeoffs across various LLM architectures and model sizes. We systematically evaluate prompt-based and model-tuning-based techniques for reducing reasoning token length while maintaining performance quality. We further profile test-time scaling methods with varying degrees of parallelism to maximize accuracy under strict latency budgets. Through these analyses, EdgeReasoning maps the Pareto frontier of achievable accuracy-latency configurations, offering systematic guidance for optimal edge deployment of reasoning LLMs.

Related papers

• PILOT: Planning via Internalized Latent Optimization Trajectories for Large Language Models [51.43746425777865]
  Large Language Models (LLMs) often lack the capacity to formulate global strategies, leading to error propagation in long-horizon tasks.<n>We propose PILOT, a framework designed to internalize the strategic oversight of large models into intrinsic Latent Guidance.
  arXiv  Detail & Related papers  (2026-01-07T12:38:56Z)
• Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models [97.55009021098554]
  This work aims to identify the key determinants of SLMs' real-device latency and offer generalizable principles and methodologies for SLM design and training.<n>We introduce a new family of hybrid SLMs, called Nemotron-Flash, which significantly advances the accuracy-efficiency frontier of state-of-the-art SLMs.
  arXiv  Detail & Related papers  (2025-11-24T08:46:36Z)
• Agentic AI Reasoning for Mobile Edge General Intelligence: Fundamentals, Approaches, and Directions [74.35421055079655]
  Large language models (LLMs) have enabled an emergence of agentic artificial intelligence (AI) with powerful reasoning and autonomous decision-making capabilities.<n>Mobile Edge General Intelligence (MEGI) brings real-time, privacy-preserving reasoning to the network edge.<n>We propose a joint optimization framework for efficient LLM reasoning deployment in MEGI.
  arXiv  Detail & Related papers  (2025-09-27T10:53:48Z)
• Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute [61.00662702026523]
  We propose a unified Test-Time Compute scaling framework that leverages increased inference-time instead of larger models.<n>Our framework incorporates two complementary strategies: internal TTC and external TTC.<n>We demonstrate our textbf32B model achieves a 46% issue resolution rate, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1.
  arXiv  Detail & Related papers  (2025-03-31T07:31:32Z)
• On Accelerating Edge AI: Optimizing Resource-Constrained Environments [1.7355861031903428]
  Resource-constrained edge deployments demand AI solutions that balance high performance with stringent compute, memory, and energy limitations.<n>We present a comprehensive overview of the primary strategies for accelerating deep learning models under such constraints.
  arXiv  Detail & Related papers  (2025-01-25T01:37:03Z)
• RE-POSE: Synergizing Reinforcement Learning-Based Partitioning and Offloading for Edge Object Detection [3.2805151494259563]
  Real-time object detection on edge devices presents significant challenges due to their limited computational resources and the high demands of deep neural network (DNN)-based detection models.<n>This paper introduces RE-POSE, a framework designed to optimize the accuracy-latency trade-off in resource-constrained edge environments.
  arXiv  Detail & Related papers  (2025-01-16T10:56:45Z)
• MoE$^2$: Optimizing Collaborative Inference for Edge Large Language Models [43.83407446438587]
  Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing tasks.<n>We introduce textitMixture-of-Edge-Experts (MoE$2$), a novel collaborative inference framework for edge LLMs.
  arXiv  Detail & Related papers  (2025-01-16T09:36:32Z)
• Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment [12.921833067052928]
  This article surveys Cognitive Edge Computing as a practical and methodical pathway for deploying reasoning-capable Large Language Models (LLMs) and autonomous AI agents on resource-constrained devices at the network edge.<n>We present a unified, cognition-preserving framework aimed at retaining multi-step reasoning under tight memory/compute budgets.<n>We synthesize advances in efficient Transformer design, multimodal integration, hardware-aware compilation, privacy-preserving learning, and agentic tool use, and map them to edge-specific operating envelopes.
  arXiv  Detail & Related papers  (2025-01-04T06:17:48Z)
• Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing [53.77822620185878]
  We propose a synergistic methodology to concurrently optimize perovskite memristor fabrication and develop robust analog DNNs.<n>We develop "BayesMulti", a training strategy utilizing BO-guided noise injection to improve the resistance of analog DNNs to memristor imperfections.<n>Our integrated approach enables use of analog computing in much deeper and wider networks, achieving up to 100-fold improvements.
  arXiv  Detail & Related papers  (2024-12-03T19:20:08Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization [20.631476379056892]
  Large Language Models (LLMs) are at the forefront of this movement. LLMs require cloud hosting, which raises issues regarding privacy, latency, and usage limitations. We present an edge intelligence optimization problem tailored for LLM inference.
  arXiv  Detail & Related papers  (2024-05-12T02:38:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.