GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization

• URL: http://arxiv.org/abs/2605.11603v1
• Date: Tue, 12 May 2026 06:32:31 GMT
• Title: GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization
• Abstract summary: Current routing methods rarely consider sustainable energy use and CO2 emissions as optimization objectives.<n>We introduce Green-Aware Routing (GAR), a constrained multi-objective optimization framework that minimizes per-request CO2 emissions.<n>GAR achieves substantial carbon reductions while maintaining competitive accuracy and p95 latency guarantees.
• Score: 3.0215547730060206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The growing deployment of large language models (LLMs) makes per-request routing essential for balancing response quality and computational cost across heterogeneous model pools. Current routing methods rarely consider sustainable energy use and CO2 emissions as optimization objectives, despite grid carbon intensity varying by time and region, and models differing significantly in energy consumption. To address this gap, we introduce Green-Aware Routing (GAR), a constrained multi-objective optimization framework that minimizes per-request CO2 emissions subject to explicit accuracy floors and p95-latency service-level objectives (SLOs). GAR employs adaptive constraint optimization through per-dataset floor tuning and incorporates lightweight estimators for correctness, tail latency, and carbon emissions, enabling real-time routing decisions without additional inference passes. We present GAR-PD, a practical online primal-dual routing algorithm for rolling carbon budgets, alongside heuristic variants that achieve high feasibility coverage while limiting accuracy degradation. Comprehensive experiments across standard NLP benchmarks with heterogeneous LLM pools (7B-70B) demonstrate that GAR achieves substantial carbon reductions while maintaining competitive accuracy and p95 latency guarantees, providing a practical, theoretically grounded approach to sustainable LLM inference.

Related papers

• Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems [0.6539757820891136]
  Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems.<n>Current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions.<n>We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture.
  arXiv  Detail & Related papers  (2026-09-11T21:51:39Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards [69.74686029941881]
  Reinforcement Learning with Verifiable Rewards (RLVR) is an effective paradigm for improving the reasoning capabilities of large language models.<n>We propose a unified neural scheduling framework that adaptively selects high-value rollouts throughout training.<n>Experiments on six mathematical reasoning benchmarks demonstrate consistent gains in performance and training efficiency across multiple RLVR optimization methods.
  arXiv  Detail & Related papers  (2026-02-09T10:51:58Z)
• Adaptive Linear Path Model-Based Diffusion [52.84663832658799]
  We introduce Linear Path Model-Based Diffusion (LP-MBD), which replaces the variance-preserving schedule with a flow-matching linear probability path.<n>We also propose Adaptive LP-MBD (ALP-MBD), which leverages reinforcement learning to adjust diffusion steps and noise levels according to task complexity and environmental conditions.
  arXiv  Detail & Related papers  (2026-02-02T21:33:03Z)
• Reinforcement Learning-Based Energy-Aware Coverage Path Planning for Precision Agriculture [6.218614094739671]
  This paper proposes an energy-aware CPP framework grounded in Soft Actor-Critic (SAC) reinforcement learning.<n>To enable robust and adaptive decision-making under energy limitations, the framework integrates Conal Neural Networks (CNNs) for spatial feature extraction.<n>A dedicated reward function is designed to jointly optimize coverage efficiency, energy consumption, and return-to-base constraints.
  arXiv  Detail & Related papers  (2026-01-23T02:33:14Z)
• Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models [23.37504394417425]
  We propose HIA (Heuristic-Guided Inference-time Alignment), a tuning-free, black-box-compatible approach that uses a lightweight prompt.<n>We find that HIA is effective under low-inference budgets with as little as one or two response queries.
  arXiv  Detail & Related papers  (2025-08-07T08:54:27Z)
• CarboFormer: A Lightweight Semantic Segmentation Architecture for Efficient Carbon Dioxide Detection Using Optical Gas Imaging [4.567122178196833]
  Carbon dioxide (CO$$) emissions are critical indicators of both environmental impact and industrial processes.<n>We introduce CarboFormer, a lightweight semantic segmentation framework for Optical Gas Imaging (OGI)<n>Our approach integrates an optimized encoder-decoder architecture with specialized multi-scale feature fusion and auxiliary supervision strategies.
  arXiv  Detail & Related papers  (2025-05-23T18:01:42Z)
• CATransformers: Carbon Aware Transformers Through Joint Model-Hardware Optimization [15.52721716387361]
  We introduce ourframework -- the first carbon-aware co-optimization framework for Transformer-based models and hardware accelerators.<n>Our results emphasize the need for holistic optimization methods that prioritize carbon efficiency without compromising model capability and execution time performance.
  arXiv  Detail & Related papers  (2025-05-02T16:49:10Z)
• AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services [14.664814078159282]
  Large language models (LLMs) have become a growing concern due to their substantial energy consumption and carbon footprint.<n>We propose AOLO, a framework for analysis and optimization for low-carbon oriented wireless LLM services.<n>AOLO introduces a comprehensive carbon footprint model that quantifies greenhouse gas emissions across the entire LLM service chain.<n>We propose a low-carbon-oriented optimization algorithm, i.e., SNN-based deep reinforcement learning (SDRL)
  arXiv  Detail & Related papers  (2025-03-06T13:21:38Z)
• Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation [58.550710456745726]
  Fine-tuning is the most effective way of adapting pre-trained large language models (LLMs) to downstream applications. Existing techniques on efficient fine-tuning can only achieve limited reduction of such FLOPs. We present GreenTrainer, a new technique that adaptively evaluates different tensors' backpropagation costs and contributions to the fine-tuned model accuracy.
  arXiv  Detail & Related papers  (2023-09-22T21:55:18Z)
• Solving Multistage Stochastic Linear Programming via Regularized Linear Decision Rules: An Application to Hydrothermal Dispatch Planning [77.34726150561087]
  We propose a novel regularization scheme for linear decision rules (LDR) based on the AdaSO (adaptive least absolute shrinkage and selection operator) Experiments show that the overfit threat is non-negligible when using the classical non-regularized LDR to solve MSLP. For the LHDP problem, our analysis highlights the following benefits of the proposed framework in comparison to the non-regularized benchmark.
  arXiv  Detail & Related papers  (2021-10-07T02:36:14Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.