Toward Sustainable Distributed LLM Inference: A Systems Synthesis and Research Agenda for an Energy-, Carbon-, and Cache-Aware llm-d Control Plane

• URL: http://arxiv.org/abs/2609.05565v1
• Date: Thu, 03 Sep 2026 21:56:54 GMT
• Title: Toward Sustainable Distributed LLM Inference: A Systems Synthesis and Research Agenda for an Energy-, Carbon-, and Cache-Aware llm-d Control Plane
• Abstract summary: This paper asks a practical engineering question: what do they imply when the decision point is a distributed inference control plane such as llm-d?<n>I group the literature into recurring design patterns and use those patterns to sketch a Sustainable Inference Control Plane (SICP) for llm-d.<n>The proposed control plane would consider latency, energy, carbon, cache reuse, serving cost, and quality when routing and scaling, while keeping TTFT/TPOT SLOs as hard constraints.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) sustainability is increasingly a serving-systems problem, not only a training problem. In production, energy and carbon impact depend on more than model size: workload shape, batching, key-value (KV) cache reuse, prefill/decode placement, model and accelerator choice, power state, geographic carbon intensity, and service-level objectives (SLOs) all matter. Recent systems papers study many of these factors separately. This paper connects those results and asks a practical engineering question: what do they imply when the decision point is a distributed inference control plane such as llm-d? The contribution here is synthesis, not a new set of benchmark results. Reported performance, energy, carbon, and cost improvements remain the results of the cited papers and systems. I group the literature into recurring design patterns and use those patterns to sketch a Sustainable Inference Control Plane (SICP) for llm-d. The proposed control plane would consider latency, energy, carbon, cache reuse, serving cost, and quality when routing and scaling, while keeping TTFT/TPOT SLOs as hard constraints. I also outline an evaluation framework based on SLO-satisfied goodput per joule and per gram CO2e, together with a reproducible experimental plan. The main observation from connecting the literature is that sustainable LLM inference is unlikely to come from one "green" model or one accelerator; it is more naturally treated as a control problem across model, phase, cache, hardware, replica, region, and time.

Related papers

• InFactPlanner: Planning Sustainable Geo-Distributed LLM Data Centers [0.15999407512883507]
  We present InFactPlanner, a trace-driven decision-support framework for what-if analysis of sustainable AI data center deployment.<n>InFactPlanner combines query traces, hardware-model profiles, candidate site configurations, PUE/WUE parameters, renewable generation models, and time-varying grid carbon intensity to estimate power, energy, carbon emissions, water use, latency, and server utilization.
  arXiv  Detail & Related papers  (2026-08-13T07:57:22Z)
• How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms [8.304081609008119]
  We compare three dominant VTG output paradigms: Text Numeral Generation, Temporal Token Generation, and Continuous Temporal Decoding.<n>Our results demonstrate that the choice of output formulation significantly affects both grounding accuracy and computational cost, independent of model scale.<n>These findings provide objective empirical guidelines for designing efficient, deployment-ready VTG systems.
  arXiv  Detail & Related papers  (2026-04-10T05:10:15Z)
• Evaluating the Environmental Impact of using SLMs and Prompt Engineering for Code Generation [2.2165845825140695]
  This paper presents the first systematic empirical study investigating how different prompt engineering strategies impact code generation accuracy alongside sustainability factors.<n>We evaluate six prominent prompting strategies across 11 open-source models (ranging from 1B to 34B parameters) using the HumanEval+ and MBPP+ benchmarks.<n>Our findings indicate that Chain-of-Thought, being a simpler prompting technique, can provide a near-optimal balance between reasoning capability and energy efficiency.
  arXiv  Detail & Related papers  (2026-04-03T06:37:51Z)
• Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs [49.66344956133349]
  Reasoning capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models.<n>This paper proposes Reasoning Palette, a novel latent-modulation framework that endows the model with a latent variable for strategic contextualization.
  arXiv  Detail & Related papers  (2025-12-19T03:32:53Z)
• A Collaborative Platform for Soil Organic Carbon Inference Based on Spatiotemporal Remote Sensing Data [3.9589674165097897]
  WALGREEN is a platform that enhances SOC inference by overcoming limitations of current applications.<n>WALGREEN generates predictive models using historical public and private data.<n>It offers a user-friendly interface for researchers, policymakers, and land managers to access carbon data, analyze trends, and support evidence-based decision-making.
  arXiv  Detail & Related papers  (2025-04-17T04:50:18Z)
• GreenDFL: a Framework for Assessing the Sustainability of Decentralized Federated Learning Systems [13.15941655109666]
  Decentralized Federated Learning (DFL) is an emerging paradigm that enables collaborative model training without centralized data and model aggregation.<n>This work aims to develop a comprehensive and operational framework for assessing the sustainability of DFL systems.<n>The proposed GreenDFL provides a comprehensive and practical approach for assessing the sustainability of DFL systems.
  arXiv  Detail & Related papers  (2025-02-27T16:27:42Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Power Hungry Processing: Watts Driving the Cost of AI Deployment? [74.19749699665216]
  generative, multi-purpose AI systems promise a unified approach to building machine learning (ML) models into technology. This ambition of generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions
  arXiv  Detail & Related papers  (2023-11-28T15:09:36Z)
• A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsification [5.341266334051207]
  Federated Learning (FL) methods adopt efficient communication technologies to distribute machine learning tasks across edge devices. This paper proposes a framework for real-time monitoring of the energy and carbon footprint impacts of FL systems.
  arXiv  Detail & Related papers  (2023-10-12T07:20:03Z)
• ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models [70.45441031021291]
  Large Vision-Language Models (LVLMs) can understand the world comprehensively by integrating rich information from different modalities. LVLMs are often problematic due to their massive computational/energy costs and carbon consumption. We propose Efficient Coarse-to-Fine LayerWise Pruning (ECoFLaP), a two-stage coarse-to-fine weight pruning approach for LVLMs.
  arXiv  Detail & Related papers  (2023-10-04T17:34:00Z)
• Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning [77.62876532784759]
  Machine learning (ML) requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the energy source. We present a survey of the carbon emissions of 95 ML models across time and different tasks in natural language processing and computer vision.
  arXiv  Detail & Related papers  (2023-02-16T18:35:00Z)
• A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning [48.63610479916003]
  This article breaks down and analyzes the main factors that influence the environmental footprint of distributed learning policies. It models both vanilla and decentralized FL policies driven by consensus. Results show that FL allows remarkable end-to-end energy savings (30%-40%) for wireless systems characterized by low bit/Joule efficiency.
  arXiv  Detail & Related papers  (2021-03-18T16:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.