On the Effectiveness of Proposed Techniques to Reduce Energy Consumption in RAG Systems: A Controlled Experiment

• URL: http://arxiv.org/abs/2601.02522v2
• Date: Sun, 11 Jan 2026 14:57:22 GMT
• Title: On the Effectiveness of Proposed Techniques to Reduce Energy Consumption in RAG Systems: A Controlled Experiment
• Abstract summary: This study presents a controlled experiment investigating five practical techniques aimed at reducing energy consumption in RAG systems.<n>We evaluated the impact of these techniques on energy consumption, latency, and accuracy.<n>We reveal that techniques such as increasing similarity retrieval thresholds, reducing embedding sizes, applying vector indexing, and using a BM25S reranker can significantly reduce energy usage, up to 60% in some cases.
• Score: 2.7757571644758166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rising energy demands of machine learning (ML), e.g., implemented in popular variants like retrieval-augmented generation (RAG) systems, have raised significant concerns about their environmental sustainability. While previous research has proposed green tactics for ML-enabled systems, their empirical evaluation within RAG systems remains largely unexplored. This study presents a controlled experiment investigating five practical techniques aimed at reducing energy consumption in RAG systems. Using a production-like RAG system developed at our collaboration partner, the Software Improvement Group, we evaluated the impact of these techniques on energy consumption, latency, and accuracy. Through a total of 9 configurations spanning over 200 hours of trials using the CRAG dataset, we reveal that techniques such as increasing similarity retrieval thresholds, reducing embedding sizes, applying vector indexing, and using a BM25S reranker can significantly reduce energy usage, up to 60% in some cases. However, several techniques also led to unacceptable accuracy decreases, e.g., by up to 30% for the indexing strategies. Notably, finding an optimal retrieval threshold and reducing embedding size substantially reduced energy consumption and latency with no loss in accuracy, making these two techniques truly energy-efficient. We present the first comprehensive, empirical study on energy-efficient design techniques for RAG systems, providing guidance for developers and researchers aiming to build sustainable RAG applications.

Related papers

• Evaluating the Environmental Impact of using SLMs and Prompt Engineering for Code Generation [2.2165845825140695]
  This paper presents the first systematic empirical study investigating how different prompt engineering strategies impact code generation accuracy alongside sustainability factors.<n>We evaluate six prominent prompting strategies across 11 open-source models (ranging from 1B to 34B parameters) using the HumanEval+ and MBPP+ benchmarks.<n>Our findings indicate that Chain-of-Thought, being a simpler prompting technique, can provide a near-optimal balance between reasoning capability and energy efficiency.
  arXiv  Detail & Related papers  (2026-04-03T06:37:51Z)
• Green LLM Techniques in Action: How Effective Are Existing Techniques for Improving the Energy Efficiency of LLM-Based Applications in Industry? [2.3683790724077864]
  The rapid adoption of large language models (LLMs) has raised concerns about their substantial energy consumption.<n>We analyzed an application in an industrial context at Schuberg Philis, a Dutch IT services company.<n>Several techniques, such as Prompt Optimization and 2-bit Quantization, managed to reduce energy use significantly, sometimes by up to 90%.<n>The only technique that achieved significant and strong energy reductions without harming the other qualities substantially was Small and Large Model Collaboration via Nvidia's Prompt Task and Complexity.
  arXiv  Detail & Related papers  (2026-01-05T19:35:29Z)
• Lightweight LSTM Model for Energy Theft Detection via Input Data Reduction [0.0]
  This paper proposes a lightweight detection unit, or watchdog mechanism, designed to act as a pre-filter.<n>It reduces the volume of input fed to the LSTM model, limiting it to instances that are more likely to involve energy theft.<n>Results indicate a power consumption reduction exceeding 64%, with minimal loss in detection accuracy and consistently high recall.
  arXiv  Detail & Related papers  (2025-06-14T14:40:03Z)
• Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights [2.1249213103048414]
  The rapid adoption of large language models (LLMs) has led to significant energy consumption and carbon emissions.<n>This paper explores the integration of energy-efficient optimization techniques in the deployment of LLMs to address these concerns.
  arXiv  Detail & Related papers  (2025-04-07T21:56:59Z)
• AI-in-the-Loop Sensing and Communication Joint Design for Edge Intelligence [65.29835430845893]
  We propose a framework that enhances edge intelligence through AI-in-the-loop joint sensing and communication.<n>A key contribution of our work is establishing an explicit relationship between validation loss and the system's tunable parameters.<n>We show that our framework reduces communication energy consumption by up to 77 percent and sensing costs measured by the number of samples by up to 52 percent.
  arXiv  Detail & Related papers  (2025-02-14T14:56:58Z)
• THOR: A Generic Energy Estimation Approach for On-Device Training [34.57867978862375]
  THOR is a generic approach for energy consumption estimation in deep neural network (DNN) training.<n>We conduct extensive experiments with various types of models across different real-world platforms.<n>The results demonstrate that THOR has effectively reduced the Mean Absolute Percentage Error (MAPE) by up to 30%.
  arXiv  Detail & Related papers  (2025-01-27T03:29:02Z)
• Prompt engineering and its implications on the energy consumption of Large Language Models [4.791072577881446]
  Large language models (LLMs) in software engineering pose severe challenges regarding computational resources, data centers, and carbon emissions.<n>In this paper, we investigate how prompt engineering techniques (PETs) can impact the carbon emission of the Llama 3 model for the code generation task.
  arXiv  Detail & Related papers  (2025-01-10T11:49:31Z)
• Multiagent Reinforcement Learning with an Attention Mechanism for Improving Energy Efficiency in LoRa Networks [52.96907334080273]
  As the network scale increases, the energy efficiency of LoRa networks decreases sharply due to severe packet collisions. We propose a transmission parameter allocation algorithm based on multiagent reinforcement learning (MALoRa) Simulation results demonstrate that MALoRa significantly improves the system EE compared with baseline algorithms.
  arXiv  Detail & Related papers  (2023-09-16T11:37:23Z)
• Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
  Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency. In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
  arXiv  Detail & Related papers  (2021-09-17T08:55:48Z)
• Demand Responsive Dynamic Pricing Framework for Prosumer Dominated Microgrids using Multiagent Reinforcement Learning [59.28219519916883]
  This paper proposes a new multiagent Reinforcement Learning based decision-making environment for implementing a Real-Time Pricing (RTP) DR technique in a prosumer dominated microgrid. The proposed technique addresses several shortcomings common to traditional DR methods and provides significant economic benefits to the grid operator and prosumers.
  arXiv  Detail & Related papers  (2020-09-23T01:44:57Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [68.37641996188133]
  We introduce a framework for tracking realtime energy consumption and carbon emissions. We create a leaderboard for energy efficient reinforcement learning algorithms. We propose strategies for mitigation of carbon emissions and reduction of energy consumption.
  arXiv  Detail & Related papers  (2020-01-31T05:12:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.