Green Prompting

• URL: http://arxiv.org/abs/2503.10666v2
• Date: Tue, 08 Apr 2025 10:56:07 GMT
• Title: Green Prompting
• Authors: Marta Adamska, Daria Smirnova, Hamid Nasiri, Zhengxin Yu, Peter Garraghan,
• Abstract summary: Large Language Models (LLMs) have become widely used across various domains spanning search engines, code generation, and text creation.<n>This study empirically study how different prompt and response characteristics directly impact LLM inference energy cost.
• Score: 2.377736044164336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have become widely used across various domains spanning search engines, code generation, and text creation. However, a major concern associated with their adoption is the high cost of inference, impacting both their sustainability and financial feasibility. In this study, we empirically study how different prompt and response characteristics directly impact LLM inference energy cost. We conduct experiments leveraging three open-source transformer-based LLMs across three task types$-$question answering, sentiment analysis, and text generation. For each inference, we analyzed prompt and response characteristics (length, semantic meaning, time taken, energy consumption). Our results demonstrate that even when presented with identical tasks, models generate responses with varying characteristics and subsequently exhibit distinct energy consumption patterns. We found that prompt length is less significant than the semantic meaning of the task itself. In addition, we identified specific keywords associated with higher or lower energy usage that vary between associated tasks. These findings highlight the importance of prompt design in optimizing inference efficiency. We conclude that the semantic meaning of prompts and certain task-related keywords significantly impact inference costs, leading the way for deeper exploration towards creating energy-adaptive LLMs.

Related papers

• LogiDynamics: Unraveling the Dynamics of Logical Inference in Large Language Model Reasoning [49.58786377307728]
  This paper adopts an exploratory approach by introducing a controlled evaluation environment for analogical reasoning. We analyze the comparative dynamics of inductive, abductive, and deductive inference pipelines. We investigate advanced paradigms such as hypothesis selection, verification, and refinement, revealing their potential to scale up logical inference.
  arXiv  Detail & Related papers  (2025-02-16T15:54:53Z)
• Benchmarking Prompt Sensitivity in Large Language Models [13.986971540998258]
  Large language Models (LLMs) are highly sensitive to variations in prompt formulation.<n>This paper introduces a new task, Prompt Sensitivity Prediction, and a dataset designed to investigate the effects of slight prompt variations on LLM performance.
  arXiv  Detail & Related papers  (2025-02-09T23:01:03Z)
• Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models [19.00048908546417]
  Large language models (LLMs) are increasingly recognized for their exceptional generative capabilities and versatility.<n>This study conducts a comprehensive benchmarking of LLM inference energy across a wide range of NLP tasks.<n>We find that quantization and optimal batch sizes, along with targeted prompt phrases, can significantly reduce energy usage.
  arXiv  Detail & Related papers  (2025-02-08T15:34:52Z)
• Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding [58.364933651703524]
  We show that concentrated massive values consistently emerge in specific regions of attention queries.<n>These massive values play a critical role in interpreting contextual knowledge.<n>We trace the emergence of massive values and find that such concentration is caused by Rotary Positional.
  arXiv  Detail & Related papers  (2025-02-03T17:47:03Z)
• Investigating Energy Efficiency and Performance Trade-offs in LLM Inference Across Tasks and DVFS Settings [1.5749416770494706]
  Large language models (LLMs) have shown significant improvements in many natural language processing (NLP) tasks.<n>LLMs are resource-intensive, requiring extensive computational resources both during training and inference.<n>As their adoption accelerates, the sustainability of LLMs has become a critical issue.
  arXiv  Detail & Related papers  (2025-01-14T16:02:33Z)
• Exploring Knowledge Boundaries in Large Language Models for Retrieval Judgment [56.87031484108484]
  Large Language Models (LLMs) are increasingly recognized for their practical applications. Retrieval-Augmented Generation (RAG) tackles this challenge and has shown a significant impact on LLMs. By minimizing retrieval requests that yield neutral or harmful results, we can effectively reduce both time and computational costs.
  arXiv  Detail & Related papers  (2024-11-09T15:12:28Z)
• Image First or Text First? Optimising the Sequencing of Modalities in Large Language Model Prompting and Reasoning Tasks [0.0]
  This paper examines how the sequencing of images and text within multi-modal prompts influences the reasoning performance of large language models (LLMs) For simpler tasks involving a single image, modality sequencing had a clear impact on accuracy. In more complex tasks involving multiple images and intricate reasoning steps, the effect of sequencing diminished, likely due to the increased cognitive demands of the task.
  arXiv  Detail & Related papers  (2024-10-04T00:55:15Z)
• QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning [58.767866109043055]
  We introduce Query-dependent Prompt Optimization (QPO), which iteratively fine-tune a small pretrained language model to generate optimal prompts tailored to the input queries. We derive insights from offline prompting demonstration data, which already exists in large quantities as a by-product of benchmarking diverse prompts on open-sourced tasks. Experiments on various LLM scales and diverse NLP and math tasks demonstrate the efficacy and cost-efficiency of our method in both zero-shot and few-shot scenarios.
  arXiv  Detail & Related papers  (2024-08-20T03:06:48Z)
• Word Importance Explains How Prompts Affect Language Model Outputs [0.7223681457195862]
  This study presents a method to improve the explainability of large language models by varying individual words in prompts. Unlike classical attention, word importance measures the impact of prompt words on arbitrarily-defined text scores. Results show that word importance scores are closely related to the expected suffix importances for multiple scoring functions.
  arXiv  Detail & Related papers  (2024-03-05T15:04:18Z)
• INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning [59.07490387145391]
  Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks. Their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. We introduce a novel instruction tuning dataset, INTERS, encompassing 20 tasks across three fundamental IR categories.
  arXiv  Detail & Related papers  (2024-01-12T12:10:28Z)
• Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering [58.82325933356066]
  Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge. We present a detailed study of how different settings affect performance for Visual Question Answering.
  arXiv  Detail & Related papers  (2022-09-30T19:12:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.