Babbling Suppression: Making LLMs Greener One Token at a Time

• URL: http://arxiv.org/abs/2604.06755v1
• Date: Wed, 08 Apr 2026 07:21:02 GMT
• Title: Babbling Suppression: Making LLMs Greener One Token at a Time
• Abstract summary: Large Language Models (LLMs) are increasingly used in modern software development.<n>LLMs produce extraneous output, referred to as "babbling", which incurs additional cognitive, economic, and energy costs.<n>This work proposes a practical, model-agnostic approach to reduce unnecessary output without compromising solution accuracy.
• Score: 46.879983975894135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context: Large Language Models (LLMs) are increasingly used in modern software development, aiding in code generation, code completion, and refactoring through AI-powered assistants. While they accelerate development workflows, they often produce extraneous output, referred to as "babbling", which incurs additional cognitive, economic, and energy costs. Objective: This work investigates the problem of babbling in LLM-based code generation and proposes a practical, model-agnostic approach to reduce unnecessary output without compromising solution accuracy. Method: We introduce Babbling Suppression (BS), a method that integrates test execution into the LLM generation process by evaluating intermediate outputs and terminating generation once a solution passes all tests. This prevents excessive token generation while having no impact on model accuracy. An empirical study was conducted across two Python and two Java benchmarks, targeting four 3-4B parameter models and six 6-7B parameter models. Results: Our findings show that babbling occurs across all tested models, with higher frequency in Java than in Python. Applying BS significantly reduces energy consumption by up to 65% for Python and 62% for Java in models prone to babbling. Across 40 model-benchmark pairs, 29 showed reduced mean energy consumption, with reductions exceeding 20% in 22 cases. Generated token count decreased in 35 pairs, while the GPU energy-per-token overhead of BS remained below 10% for 26 pairs, decreased for 2, and reached a maximum of 24%, yielding net energy savings in most cases. Implications: BS can make AI-assisted programming more efficient and sustainable by reducing energy consumption and minimizing cognitive effort by developers. Its model-agnostic design allows easy integration, suggesting broad applicability.

Related papers

• Towards Green AI: Decoding the Energy of LLM Inference in Software Development [46.879983975894135]
  AI-assisted tools are increasingly integrated into software development, but their reliance on large language models (LLMs) introduces substantial computational and energy costs.<n>We conduct a phase-level analysis of LLM inference energy consumption, distinguishing between the (1) prefill, where the model processes the input and builds internal representations, and (2) decoding, where output tokens are generated using the stored state.
  arXiv  Detail & Related papers  (2026-02-05T14:38:19Z)
• A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models [53.31664844941449]
  ProActive Self-Refinement (PASR) is a novel method for improving large language models (LLMs)<n>Unlike methods that regenerate entire responses, PASR proactively decides whether, when, and how to refine based on the model's internal state and evolving context.<n>We conduct extensive experiments on a diverse set of 10 tasks to evaluate the effectiveness of PASR.
  arXiv  Detail & Related papers  (2025-08-18T13:07:21Z)
• Accelerated Test-Time Scaling with Model-Free Speculative Sampling [58.69141724095398]
  We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach.<n>We show that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding.<n>As a model-free approach, STAND can be applied to any existing language model without additional training.
  arXiv  Detail & Related papers  (2025-06-05T07:31:18Z)
• Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering [51.7496756448709]
  Language models (LMs) perform well on coding benchmarks but struggle with real-world software engineering tasks.<n>Existing approaches rely on supervised fine-tuning with high-quality data, which is expensive to curate at scale.<n>We propose Test-Time Scaling (EvoScale), a sample-efficient method that treats generation as an evolutionary process.
  arXiv  Detail & Related papers  (2025-05-29T16:15:36Z)
• Scalable LLM Math Reasoning Acceleration with Low-rank Distillation [57.922185576872444]
  We propose a resource-efficient distillation method to recover lost capabilities from deploying efficient inference methods.<n>With original weights unperturbed, roughly 1% of additional parameters, and only 20K synthetic training samples, we are able to recover much if not all of the math capabilities lost.
  arXiv  Detail & Related papers  (2025-05-08T17:51:24Z)
• Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models [8.70160958177614]
  Program synthesis with Large Language Models (LLMs) suffers from a "near-miss syndrome"<n>We address this with a multi-agent framework called Synthesize, Execute, Instruct, Debug, and Repair (SEIDR)<n>We empirically explore these trade-offs by comparing replace-focused, repair-focused, and hybrid debug strategies.
  arXiv  Detail & Related papers  (2025-03-10T16:56:51Z)
• Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding [48.55176091771745]
  Test-time scaling enhances large language model performance by allocating additional compute resources during inference.<n>Best-of-N (BoN) sampling serves as a common sampling-based scaling technique.<n>We propose Self-Truncation Best-of-N (ST-BoN), a decoding method that avoids fully generating all N samples.<n>In terms of cost, ST-BoN achieves the same performance as Full-BoN while saving computational cost by 70%-80%, and under the same cost, it can improve accuracy by 3-4 points.
  arXiv  Detail & Related papers  (2025-03-03T11:21:01Z)
• Token-by-Token Regeneration and Domain Biases: A Benchmark of LLMs on Advanced Mathematical Problem-Solving [0.0]
  This study evalu-ates 10 large language models (LLMs) with 7 to 8 billion parameters using the MATH dataset.<n>The focus is on their ability to generate executable Python code as a step in their reasoning process, involving over 9,450 code executions.
  arXiv  Detail & Related papers  (2025-01-28T17:11:36Z)
• ALPINE: An adaptive language-agnostic pruning method for language models for code [4.148857672591562]
  Language models of code have demonstrated state-of-the-art performance across various software engineering and source code analysis tasks.<n>This work introduces ALPINE, an adaptive programming language-agnostic pruning technique designed to substantially reduce these models' computational overhead.<n>Experiments on two software engineering tasks, defect prediction and code clone detection across three language models CodeBERT, GraphCodeBERT and UniXCoder show that ALPINE achieves up to a 50% reduction in FLOPs, a 58.1% decrease in memory footprint, and a 28.1% improvement in throughput on average.
  arXiv  Detail & Related papers  (2024-07-04T20:42:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.