Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems

• URL: http://arxiv.org/abs/2609.13559v1
• Date: Fri, 11 Sep 2026 21:51:39 GMT
• Title: Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems
• Abstract summary: Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems.<n>Current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions.<n>We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture.
• Score: 0.6539757820891136
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems. Nevertheless, current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions. We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture, combining edge and cloud LLMs on heterogeneous hardware. At its core, a lightweight k-NN predictor operating in a unified semantic-lexical embedding space estimates query-specific accuracy, delay, and power consumption on each edge tier. These predictions are then combined with real-time grid carbon intensity to route every query to the lowest-emission tier capable of executing it successfully. Evaluated on state-of-the-art function-calling benchmarks and LLM families, our framework matches cloud-level accuracy while reducing operational carbon emissions by $4\times$ on average.

Related papers

• GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization [3.0215547730060206]
  Current routing methods rarely consider sustainable energy use and CO2 emissions as optimization objectives.<n>We introduce Green-Aware Routing (GAR), a constrained multi-objective optimization framework that minimizes per-request CO2 emissions.<n>GAR achieves substantial carbon reductions while maintaining competitive accuracy and p95 latency guarantees.
  arXiv  Detail & Related papers  (2026-05-12T06:32:31Z)
• SGA-MCTS: Decoupling Planning from Execution via Training-Free Atomic Experience Retrieval [74.1918709002557]
  We introduce textbfSGA-MCTS, a framework that casts LLM planning as non-parametric retrieval.<n>Online, a retrieval-augmented agent employs a hybrid symbolic-semantic mechanism to fetch relevant State-Goal-Action atoms.<n>By effectively amortizing the heavy computational cost of search, SGA-MCTS achieves System 2 reasoning depth at System 1 inference speeds, rendering autonomous planning both scalable and real-time feasible.
  arXiv  Detail & Related papers  (2026-04-16T07:22:36Z)
• ROSA-Tuning: Enhancing Long-Context Modeling via Suffix Matching [10.500711586744746]
  Long-context capability and computational efficiency are among the central challenges facing today's large language models.<n>This paper proposes ROSA-Tuning, a retrieval-and-recall mechanism for enhancing the long-context modeling ability of pretrained models.
  arXiv  Detail & Related papers  (2026-01-14T13:12:08Z)
• GLOW: Graph-Language Co-Reasoning for Agentic Workflow Performance Prediction [51.83437071408662]
  We propose GLOW, a unified framework for AW performance prediction.<n>GLOW combines the graph-structure modeling capabilities of GNNs with the reasoning power of LLMs.<n>Experiments on FLORA-Bench show that GLOW outperforms state-of-the-art baselines in prediction accuracy and ranking utility.
  arXiv  Detail & Related papers  (2025-12-11T13:30:46Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices [0.44784055850794474]
  Large Language Models (LLMs) enable real-time function calling in edge AI systems but introduce significant computational overhead, leading to high power consumption and carbon emissions.<n>We introduce CarbonCall, a sustainability-aware function-calling framework that integrates dynamic tool selection, carbon-aware execution, and quantized adaptation.<n>Experiments on an NVIDIA Jetson AGX Orin show that CarbonCall reduces carbon emissions by up to 52%, power consumption by 30%, and execution time by 30%, while maintaining high efficiency.
  arXiv  Detail & Related papers  (2025-04-29T01:37:08Z)
• PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning [54.99373314906667]
  Self-supervised representation learning for point cloud has demonstrated effectiveness in improving pre-trained model performance across diverse tasks.<n>As pre-trained models grow in complexity, fully fine-tuning them for downstream applications demands substantial computational and storage resources.<n>We propose PointLoRA, a simple yet effective method that combines low-rank adaptation (LoRA) with multi-scale token selection to efficiently fine-tune point cloud models.
  arXiv  Detail & Related papers  (2025-04-22T16:41:21Z)
• AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services [14.664814078159282]
  Large language models (LLMs) have become a growing concern due to their substantial energy consumption and carbon footprint.<n>We propose AOLO, a framework for analysis and optimization for low-carbon oriented wireless LLM services.<n>AOLO introduces a comprehensive carbon footprint model that quantifies greenhouse gas emissions across the entire LLM service chain.<n>We propose a low-carbon-oriented optimization algorithm, i.e., SNN-based deep reinforcement learning (SDRL)
  arXiv  Detail & Related papers  (2025-03-06T13:21:38Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• LACS: Learning-Augmented Algorithms for Carbon-Aware Resource Scaling with Uncertain Demand [1.423958951481749]
  This paper studies the online carbon-aware resource scaling problem with unknown job lengths (OCSU) We propose LACS, a theoretically robust learning-augmented algorithm that solves OCSU. LACS achieves a 32% reduction in carbon footprint compared to the deadline-aware carbon-agnostic execution of the job.
  arXiv  Detail & Related papers  (2024-03-29T04:54:22Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.