Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency

• URL: http://arxiv.org/abs/2504.15989v2
• Date: Thu, 29 May 2025 18:29:37 GMT
• Title: Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency
• Authors: Junwei Hu, Weicheng Zheng, Yihan Liu, Yan Liu,
• Abstract summary: Chain of Thought (CoT) reasoning has become an essential approach for automated code repair.<n>CoT leads to substantial increases in token consumption, reducing inference efficiency and raising computational costs.<n>This paper proposes three targeted optimization strategies: Context Awareness, Responsibility Tuning, and Cost Sensitive.
• Score: 5.044393644778693
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing adoption of large language models (LLMs) in software engineering, the Chain of Thought (CoT) reasoning paradigm has become an essential approach for automated code repair. However, the explicit multi-step reasoning in CoT leads to substantial increases in token consumption, reducing inference efficiency and raising computational costs, especially for complex code repair tasks. Most prior research has focused on improving the correctness of code repair while largely overlooking the resource efficiency of the reasoning process itself. To address this challenge, this paper proposes three targeted optimization strategies: Context Awareness, Responsibility Tuning, and Cost Sensitive. Context Awareness guides the model to focus on key contextual information, Responsibility Tuning refines the structure of the reasoning process through clearer role and responsibility assignment, and Cost Sensitive incorporates resource-awareness to suppress unnecessary token generation during inference. Experiments across diverse code repair scenarios demonstrate that these methods can significantly reduce token consumption in CoT-based reasoning without compromising repair quality. This work provides novel insights and methodological guidance for enhancing the efficiency of LLM-driven code repair tasks in software engineering.

Related papers

• Is Quantization a Deal-breaker? Empirical Insights from Large Code Models [7.182449176083625]
  We apply Activation-aware Weight Quantization (AWQ) to two widely used code models, CodeLlama and DeepSeekCoder, to generate Java and Python code.<n>Our findings reveal that quantization is a robust technique that not only preserves functional correctness, but also retains key qualitative code attributes sought after by developers.
  arXiv  Detail & Related papers  (2025-07-13T14:58:19Z)
• $T^3$: Multi-level Tree-based Automatic Program Repair with Large Language Models [9.334654320797975]
  This study systematically evaluates the performance of several common CoT techniques in APR tasks.<n>It proposes an innovative framework $T3$, which integrates the powerful reasoning capabilities of LLMs with tree search.
  arXiv  Detail & Related papers  (2025-06-26T13:04:28Z)
• ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research [53.736407871322314]
  We introduce ORMind, a cognitive-inspired framework that enhances optimization through counterfactual reasoning.<n>Our approach emulates human cognition, implementing an end-to-end workflow that transforms requirements into mathematical models and executable code.<n>It is currently being tested internally in Lenovo's AI Assistant, with plans to enhance optimization capabilities for both business and consumer customers.
  arXiv  Detail & Related papers  (2025-06-02T05:11:21Z)
• Towards Effective Code-Integrated Reasoning [89.47213509714578]
  We investigate code-integrated reasoning, where models generate code when necessary and integrate feedback by executing it through a code interpreter.<n>Tool-augmented reinforcement learning can still suffer from potential instability in the learning dynamics.<n>We develop enhanced training strategies that balance exploration and stability, progressively building tool-use capabilities while improving reasoning performance.
  arXiv  Detail & Related papers  (2025-05-30T11:30:18Z)
• AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks [4.757470449749877]
  This paper introduces Adversarial Knowledge Distillation (AKD) to distill the capabilities of larger models into smaller, more efficient ones.<n>AKD provides a framework for enhancing model robustness, reliability, and security while improving their parameter-efficiency.
  arXiv  Detail & Related papers  (2025-05-05T22:41:19Z)
• Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs [53.00384299879513]
  In large language models (LLMs), code and reasoning reinforce each other.<n>Code provides verifiable execution paths, enforces logical decomposition, and enables runtime validation.<n>We identify key challenges and propose future research directions to strengthen this synergy.
  arXiv  Detail & Related papers  (2025-02-26T18:55:42Z)
• LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness [38.399282089600284]
  Large Language Models (LLMs) have demonstrated impressive performance in code generation.<n>tool: ulineLarge ulineLanguage ulineModel for Code ulineEfficiency is a novel framework that enables LLMs to generate code that balances both efficiency and correctness.
  arXiv  Detail & Related papers  (2025-02-17T07:01:18Z)
• Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability [3.9122242678047456]
  Large language models (LLMs) face significant token efficiency bottlenecks in code generation and logical reasoning tasks.<n>This paper proposes a formal framework based on symbolic compression,integrating logic, information-theoretic optimal encoding, and context-aware inference techniques.
  arXiv  Detail & Related papers  (2025-01-30T06:40:52Z)
• Generating refactored code accurately using reinforcement learning [3.179831861897336]
  We propose a novel reinforcement learning-based approach for fine-tuning and aligning code language models to perform automated, intelligent extract method on Java source code. Our approach fine-tunes sequence-to-sequence generative models and aligns them using the Proximal Policy Optimization (PPO) algorithm. Our experiments demonstrate that our approach significantly enhances the performance of large language models in code.
  arXiv  Detail & Related papers  (2024-12-23T23:09:48Z)
• Less is More: Towards Green Code Large Language Models via Unified Structural Pruning [27.428983811427827]
  We propose Flab-Pruner, an innovative unified structural pruning method that combines vocabulary, layer, and Feed-Forward Network (FFN) pruning.<n>The results demonstrate that Flab-Pruner retains 97% of the original performance after pruning 22% of the parameters and achieves the same or even better performance after post-training.
  arXiv  Detail & Related papers  (2024-12-20T14:13:09Z)
• A Theoretical Perspective for Speculative Decoding Algorithm [60.79447486066416]
  One effective way to accelerate inference is emphSpeculative Decoding, which employs a small model to sample a sequence of draft tokens and a large model to validate. This paper tackles this gap by conceptualizing the decoding problem via markov chain abstraction and studying the key properties, emphoutput quality and inference acceleration, from a theoretical perspective.
  arXiv  Detail & Related papers  (2024-10-30T01:53:04Z)
• SwiftCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning [17.355845751737423]
  Current methods primarily focus on correctness, often overlooking efficiency.<n>dataset offers a scalable and effective solution for advancing AI-driven code generation.
  arXiv  Detail & Related papers  (2024-10-14T07:05:51Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• CodeDPO: Aligning Code Models with Self Generated and Verified Source Code [52.70310361822519]
  We propose CodeDPO, a framework that integrates preference learning into code generation to improve two key code preference factors: code correctness and efficiency. CodeDPO employs a novel dataset construction method, utilizing a self-generation-and-validation mechanism that simultaneously generates and evaluates code and test cases.
  arXiv  Detail & Related papers  (2024-10-08T01:36:15Z)
• Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification [52.095460362197336]
  Large language models (LLMs) struggle with consistent and accurate reasoning. LLMs are trained primarily on correct solutions, reducing their ability to detect and learn from errors. We propose a novel collaborative method integrating Chain-of-Thought (CoT) and Program-of-Thought (PoT) solutions for verification.
  arXiv  Detail & Related papers  (2024-10-05T05:21:48Z)
• Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning [94.76546523689113]
  We introduce CodePlan, a framework that generates and follows textcode-form plans -- pseudocode that outlines high-level, structured reasoning processes. CodePlan effectively captures the rich semantics and control flows inherent to sophisticated reasoning tasks. It achieves a 25.1% relative improvement compared with directly generating responses.
  arXiv  Detail & Related papers  (2024-09-19T04:13:58Z)
• Retrieval-Augmented Instruction Tuning for Automated Process Engineering Calculations : A Tool-Chaining Problem-Solving Framework with Attributable Reflection [0.0]
  We introduce a novel autonomous agent framework leveraging Retrieval-Augmented Instruction-Tuning (RAIT) to enhance open, customizable small code language models (SLMs) By combining instruction tuned code SLMs with Retrieval-Augmented Code Generation (RACG) using external tools, the agent generates, debugs, and optimize code from natural language specifications. Our approach addresses the limitations of the current lack of a foundational AI model for specialized process engineering tasks and offers benefits of explainability, knowledge editing, and cost-effectiveness.
  arXiv  Detail & Related papers  (2024-08-28T15:33:47Z)
• Measuring Code Efficiency Optimization Capabilities with ACEOB [7.4056083791645495]
  We conduct an in-depth analysis of "code patterns" in the model training dataset, meticulously exploring human-written code. We introduce the Automatic Code Efficiency Optimization Benchmark (ACEOB), which consists of 95,359 pairs of efficient-inefficient code. To our knowledge, ACEOB is the first dataset specifically targeting Python code efficiency optimization.
  arXiv  Detail & Related papers  (2024-08-23T10:10:37Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Factor Graph Optimization of Error-Correcting Codes for Belief Propagation Decoding [62.25533750469467]
  Low-Density Parity-Check (LDPC) codes possess several advantages over other families of codes. The proposed approach is shown to outperform the decoding performance of existing popular codes by orders of magnitude.
  arXiv  Detail & Related papers  (2024-06-09T12:08:56Z)
• Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective [85.48043537327258]
  We propose MANGO (comMents As Natural loGic pivOts), including a comment contrastive training strategy and a corresponding logical comment decoding strategy. Results indicate that MANGO significantly improves the code pass rate based on the strong baselines. The robustness of the logical comment decoding strategy is notably higher than the Chain-of-thoughts prompting.
  arXiv  Detail & Related papers  (2024-04-11T08:30:46Z)
• Static Code Analysis in the AI Era: An In-depth Exploration of the Concept, Function, and Potential of Intelligent Code Analysis Agents [2.8686437689115363]
  We introduce the Intelligent Code Analysis Agent (ICAA), a novel concept combining AI models, engineering process designs, and traditional non-AI components. We observed a substantial improvement in bug detection accuracy, reducing the false-positive rate to 66% from the baseline's 85%, and a promising recall rate of 60.8%. Despite this challenge, our findings suggest that the ICAA holds considerable potential to revolutionize software quality assurance.
  arXiv  Detail & Related papers  (2023-10-13T03:16:58Z)
• A Transformer-based Approach for Source Code Summarization [86.08359401867577]
  We learn code representation for summarization by modeling the pairwise relationship between code tokens. We show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin.
  arXiv  Detail & Related papers  (2020-05-01T23:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.