How Small is Enough? Empirical Evidence of Quantized Small Language Models for Automated Program Repair

• URL: http://arxiv.org/abs/2508.16499v1
• Date: Fri, 22 Aug 2025 16:25:08 GMT
• Title: How Small is Enough? Empirical Evidence of Quantized Small Language Models for Automated Program Repair
• Authors: Kazuki Kusama, Honglin Shu, Masanari Kondo, Yasutaka Kamei,
• Abstract summary: Small language models (SLMs) perform well even with limited computational resources compared to large language models (LLMs)<n>Int8 quantization had minimal effect on APR accuracy while significantly reducing memory requirements.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: Large language models (LLMs) have greatly improved the accuracy of automated program repair (APR) methods. However, LLMs are constrained by high computational resource requirements. Aims: We focus on small language models (SLMs), which perform well even with limited computational resources compared to LLMs. We aim to evaluate whether SLMs can achieve competitive performance in APR tasks. Method: We conducted experiments on the QuixBugs benchmark to compare the bug-fixing accuracy of SLMs and LLMs. We also analyzed the impact of int8 quantization on APR performance. Results: The latest SLMs can fix bugs as accurately as--or even more accurately than--LLMs. Also, int8 quantization had minimal effect on APR accuracy while significantly reducing memory requirements. Conclusions: SLMs present a viable alternative to LLMs for APR, offering competitive accuracy with lower computational costs, and quantization can further enhance their efficiency without compromising effectiveness.

Related papers

• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads [104.9566359759396]
  We propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores.<n>Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification.
  arXiv  Detail & Related papers  (2025-11-09T03:38:29Z)
• Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance [31.38160018745285]
  Small language models (SLMs) present a more accessible alternative to large language models (LLMs)<n>This paper presents a comprehensive evaluation of 19 SLMs for news summarization across 2,000 news samples.
  arXiv  Detail & Related papers  (2025-02-02T03:07:45Z)
• Adaptive Pruning for Large Language Models with Structural Importance Awareness [66.2690963378878]
  Large language models (LLMs) have significantly improved language understanding and generation capabilities.<n>LLMs are difficult to deploy on resource-constrained edge devices due to their high computational and storage resource demands.<n>We propose structurally-aware adaptive pruning (SAAP) to significantly reduce the computational and memory costs while maintaining model performance.
  arXiv  Detail & Related papers  (2024-12-19T18:08:04Z)
• CorrectionLM: Self-Corrections with SLM for Dialogue State Tracking [16.057622631156164]
  Large language models (LLMs) have demonstrated self-improvement capabilities via feedback and refinement, but current small language models (SLMs) have had limited success in this area. We introduce CORRECTIONLM, a novel correction framework that enables SLMs to self-correct using in-context exemplars without LLM involvement.
  arXiv  Detail & Related papers  (2024-10-23T18:27:16Z)
• VinePPO: Refining Credit Assignment in RL Training of LLMs [66.80143024475635]
  We propose VinePPO, a straightforward approach that leverages the flexibility of language environments to compute unbiased Monte Carlo-based estimates.<n>Our method consistently outperforms PPO and other baselines across MATH and GSM8K datasets in less wall-clock time.
  arXiv  Detail & Related papers  (2024-10-02T15:49:30Z)
• Q-Sparse: All Large Language Models can be Fully Sparsely-Activated [93.45300714803429]
  We introduce Q-Sparse, a simple yet effective approach to training sparsely-activated large language models (LLMs) Q-Sparse enables full sparsity of activations in LLMs which can bring significant efficiency gains in inference. We also introduce Block Q-Sparse for batch training and inference.
  arXiv  Detail & Related papers  (2024-07-15T17:59:29Z)
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning [53.6472920229013]
  Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. LLMs are prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. We introduce Q*, a framework for guiding LLMs decoding process with deliberative planning.
  arXiv  Detail & Related papers  (2024-06-20T13:08:09Z)
• The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs [2.6968321526169503]
  Post-training quantization reduces the computational demand of Large Language Models (LLMs) but can weaken some of their capabilities. This paper explores how quantization affects smaller LLMs' ability to perform retrieval-augmented generation (RAG) Our findings reveal that if a 7B LLM performs the task well, quantization does not impair its performance and long-context reasoning capabilities.
  arXiv  Detail & Related papers  (2024-06-10T08:23:52Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [68.29746557968107]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans.<n> Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)
• LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit [55.73370804397226]
  Quantization, a key compression technique, can effectively mitigate these demands by compressing and accelerating large language models. We present LLMC, a plug-and-play compression toolkit, to fairly and systematically explore the impact of quantization. Powered by this versatile toolkit, our benchmark covers three key aspects: calibration data, algorithms (three strategies), and data formats.
  arXiv  Detail & Related papers  (2024-05-09T11:49:05Z)
• IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact [46.32830393597601]
  Large language models (LLMs) excel in natural language processing but demand intensive computation. This paper unveils a previously overlooked type of outliers in LLMs. We propose IntactKV to generate the KV cache of pivot tokens losslessly from the full-precision model.
  arXiv  Detail & Related papers  (2024-03-02T16:05:26Z)
• A Comprehensive Evaluation of Quantization Strategies for Large Language Models [42.03804933928227]
  Increasing the number of parameters in large language models (LLMs) usually improves performance in downstream tasks but raises compute and memory costs. Quantization techniques, which reduce the bits needed for model weights or activations with minimal performance loss, have become popular. We propose a structured evaluation framework consisting of three critical dimensions: knowledge & capacity, (2) alignment, and (3) efficiency.
  arXiv  Detail & Related papers  (2024-02-26T17:45:36Z)
• Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt [96.24800696597707]
  We introduce a new perspective to optimize this trade-off by prompting compressed models. We propose a soft prompt learning method where we expose the compressed model to the prompt learning process. Our experimental analysis suggests our soft prompt strategy greatly improves the performance of the 8x compressed LLaMA-7B model.
  arXiv  Detail & Related papers  (2023-05-17T20:45:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.