CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices

• URL: http://arxiv.org/abs/2504.20348v1
• Date: Tue, 29 Apr 2025 01:37:08 GMT
• Title: CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices
• Authors: Varatheepan Paramanayakam, Andreas Karatzas, Iraklis Anagnostopoulos, Dimitrios Stamoulis,
• Abstract summary: Large Language Models (LLMs) enable real-time function calling in edge AI systems but introduce significant computational overhead, leading to high power consumption and carbon emissions.<n>We introduce CarbonCall, a sustainability-aware function-calling framework that integrates dynamic tool selection, carbon-aware execution, and quantized adaptation.<n>Experiments on an NVIDIA Jetson AGX Orin show that CarbonCall reduces carbon emissions by up to 52%, power consumption by 30%, and execution time by 30%, while maintaining high efficiency.
• Score: 0.44784055850794474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) enable real-time function calling in edge AI systems but introduce significant computational overhead, leading to high power consumption and carbon emissions. Existing methods optimize for performance while neglecting sustainability, making them inefficient for energy-constrained environments. We introduce CarbonCall, a sustainability-aware function-calling framework that integrates dynamic tool selection, carbon-aware execution, and quantized LLM adaptation. CarbonCall adjusts power thresholds based on real-time carbon intensity forecasts and switches between model variants to sustain high tokens-per-second throughput under power constraints. Experiments on an NVIDIA Jetson AGX Orin show that CarbonCall reduces carbon emissions by up to 52%, power consumption by 30%, and execution time by 30%, while maintaining high efficiency.

Related papers

• Carbon-Efficient 3D DNN Acceleration: Optimizing Performance and Sustainability [7.059399419316343]
  3D integration improves performance but introduces sustainability challenges.<n>We propose a carbon-efficient design methodology for 3D accelerators.<n>Our approach effectively reduces silicon area and fabrication overhead while maintaining high computational accuracy.
  arXiv  Detail & Related papers  (2025-04-14T03:48:37Z)
• Ecomap: Sustainability-Driven Optimization of Multi-Tenant DNN Execution on Edge Servers [0.44784055850794474]
  This paper introduces Ecomap, a framework that adjusts the maximum power threshold of edge devices based on real-time carbon intensity.<n> Experimental results using NVIDIA Jetson AGX Xavier demonstrate that Ecomap reduces carbon emissions by an average of 30%.
  arXiv  Detail & Related papers  (2025-03-06T06:56:51Z)
• CEGI: Measuring the trade-off between efficiency and carbon emissions for SLMs and VLMs [0.0]
  This paper analyzes the performance of Small Language Models (SLMs) and Vision Language Models (VLMs)<n>To quantify the trade-off between model performance and carbon emissions, we introduce a novel metric called CEGI (Carbon Efficient Gain Index)<n>Our findings suggest that the marginal gains in accuracy from larger models do not justify the substantial increase in carbon emissions.
  arXiv  Detail & Related papers  (2024-12-03T17:32:47Z)
• The Sunk Carbon Fallacy: Rethinking Carbon Footprint Metrics for Effective Carbon-Aware Scheduling [2.562727244613512]
  We evaluate carbon-aware job scheduling and placement on a given set of servers for a number of carbon accounting metrics. We study the factors that affect the added carbon cost of such suboptimal decision-making.
  arXiv  Detail & Related papers  (2024-10-19T12:23:59Z)
• Carbon Market Simulation with Adaptive Mechanism Design [55.25103894620696]
  A carbon market is a market-based tool that incentivizes economic agents to align individual profits with the global utility. We propose an adaptive mechanism design framework, simulating the market using hierarchical, model-free multi-agent reinforcement learning (MARL) Numerical results show MARL enables government agents to balance productivity, equality, and carbon emissions.
  arXiv  Detail & Related papers  (2024-06-12T05:08:51Z)
• Generative AI for Low-Carbon Artificial Intelligence of Things with Large Language Models [67.0243099823109]
  Generative AI (GAI) holds immense potential to reduce carbon emissions of Artificial Intelligence of Things (AIoT) In this article, we explore the potential of GAI for carbon emissions reduction and propose a novel GAI-enabled solution for low-carbon AIoT. We propose a Large Language Model (LLM)-enabled carbon emission optimization framework, in which we design pluggable LLM and Retrieval Augmented Generation (RAG) modules.
  arXiv  Detail & Related papers  (2024-04-28T05:46:28Z)
• Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation [82.85015548989223]
  Pentathlon is a benchmark for holistic and realistic evaluation of model efficiency. Pentathlon focuses on inference, which accounts for a majority of the compute in a model's lifecycle. It incorporates a suite of metrics that target different aspects of efficiency, including latency, throughput, memory overhead, and energy consumption.
  arXiv  Detail & Related papers  (2023-07-19T01:05:33Z)
• Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning [77.62876532784759]
  Machine learning (ML) requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the energy source. We present a survey of the carbon emissions of 95 ML models across time and different tasks in natural language processing and computer vision.
  arXiv  Detail & Related papers  (2023-02-16T18:35:00Z)
• Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model [72.65502770895417]
  We quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes ofcarboneqif we consider only the dynamic power consumption. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of machine learning models.
  arXiv  Detail & Related papers  (2022-11-03T17:13:48Z)
• Real-time high-resolution CO$_2$ geological storage prediction using nested Fourier neural operators [58.728312684306545]
  Carbon capture and storage (CCS) plays an essential role in global decarbonization. Scaling up CCS deployment requires accurate and high-resolution modeling of the storage reservoir pressure buildup and the gaseous plume migration. We introduce Nested Fourier Neural Operator (FNO), a machine-learning framework for high-resolution dynamic 3D CO2 storage modeling at a basin scale.
  arXiv  Detail & Related papers  (2022-10-31T04:04:03Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)
• Curb Your Carbon Emissions: Benchmarking Carbon Emissions in Machine Translation [0.0]
  We study the carbon efficiency and look for alternatives to reduce the overall environmental impact of training models. In our work, we assess the performance of models for machine translation, across multiple language pairs. We examine the various components of these models to analyze aspects of our pipeline that can be optimized to reduce these carbon emissions.
  arXiv  Detail & Related papers  (2021-09-26T12:30:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.