Capturing the Effects of Quantization on Trojans in Code LLMs

• URL: http://arxiv.org/abs/2505.14200v1
• Date: Tue, 20 May 2025 11:01:14 GMT
• Title: Capturing the Effects of Quantization on Trojans in Code LLMs
• Authors: Aftab Hussain, Sadegh AlMahdi Kazemi Zarkouei, Md Rafiqul Islam Rabin, Mohammad Amin Alipour, Sen Lin, Bowen Xu,
• Abstract summary: We investigate the impact of quantization on the risk of data poisoning attacks on large language models of code.<n>We find that quantization has differing effects on code-generating LLMs.<n>We introduce a new metric for measuring trojan signals in compromised models.
• Score: 12.814581766967047
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models of code exhibit high capability in performing diverse software engineering tasks, such as code translation, defect detection, text-to-code generation, and code summarization. While their ability to enhance developer productivity has spurred widespread use, these models have also seen substantial growth in size, often reaching billions of parameters. This scale demands efficient memory resource usage, prompting practitioners to use optimization techniques such as model quantization. Quantization uses smaller bit representations for the model parameters, reducing the precision of the weights. In this work, we investigate the impact of quantization on the risk of data poisoning attacks on these models, specifically examining whether it mitigates or exacerbates such vulnerabilities. We focus on two large language models, Meta's Llama-2-7b and CodeLlama-7b, applied to an SQL code generation task. Additionally, we introduce a new metric for measuring trojan signals in compromised models. We find that quantization has differing effects on code-generating LLMs: while reducing precision does not significantly alter Llama-2's behavior, it boosts performance and reduces attack success rates in CodeLlama, particularly at 4-bit precision.

Related papers

• Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design [34.04231165571518]
  Speculative decoding and quantization effectively accelerate memory-bound inference of large language models.<n>Quantization achieves this by compressing weights and activations into lower bit-widths and also reduces computations via low-bit matrix multiplications.<n>Experiments show that the memory benefits from 4-bit weight quantization are diminished by the computational load from speculative decoding.
  arXiv  Detail & Related papers  (2025-05-28T09:55:08Z)
• Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics [64.62231094774211]
  Statefuls (e.g., Adam) maintain auxiliary information even 2x the model size in order to achieve optimal convergence.<n>SOLO enables Adam-styles to maintain quantized states with precision as low as 3 bits, or even 2 bits.<n>SOLO can thus be seamlessly applied to Adam-styles, leading to substantial memory savings with minimal accuracy loss.
  arXiv  Detail & Related papers  (2025-05-01T06:47:45Z)
• Quantizing Large Language Models for Code Generation: A Differentiated Replication [51.85505914274633]
  Large Language Models (LLMs) have shown an impressive capability in code generation and, specifically, to automatically implement requirements described in natural language.<n>LLMs pose significant challenges related to their memory (and, consequently, carbon) footprint.<n>New frontier for LLM quantization is 4-bit precision, resulting in an average memory footprint reduction of 70%.
  arXiv  Detail & Related papers  (2025-03-10T09:26:08Z)
• Precision or Peril: Evaluating Code Quality from Quantized Large Language Models [0.5249805590164902]
  Quantization has emerged as a way to mitigate the memory overhead of Large Language Models. This study aims to evaluate the current code generation capabilities of smaller LLMs using various metrics.
  arXiv  Detail & Related papers  (2024-11-16T01:31:29Z)
• M2CVD: Enhancing Vulnerability Semantic through Multi-Model Collaboration for Code Vulnerability Detection [52.4455893010468]
  Large Language Models (LLMs) have strong capabilities in code comprehension, but fine-tuning costs and semantic alignment issues limit their project-specific optimization. Code models such CodeBERT are easy to fine-tune, but it is often difficult to learn vulnerability semantics from complex code languages. This paper introduces the Multi-Model Collaborative Vulnerability Detection approach (M2CVD) to improve the detection accuracy of code models.
  arXiv  Detail & Related papers  (2024-06-10T00:05:49Z)
• Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models [1.401463252785724]
  Low-rank decomposition can be a promising direction for LLM-based applications that require real-time service at scale. We formalize the low-rank decomposition design space and show that the decomposition design space is enormous. Our results show that we can achieve a 9% model size reduction with minimal accuracy drops.
  arXiv  Detail & Related papers  (2024-05-10T17:40:02Z)
• LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit [55.73370804397226]
  Quantization, a key compression technique, can effectively mitigate these demands by compressing and accelerating large language models. We present LLMC, a plug-and-play compression toolkit, to fairly and systematically explore the impact of quantization. Powered by this versatile toolkit, our benchmark covers three key aspects: calibration data, algorithms (three strategies), and data formats.
  arXiv  Detail & Related papers  (2024-05-09T11:49:05Z)
• Quantifying the Capabilities of LLMs across Scale and Precision [12.879551933541345]
  This study investigates the effect of model scale and quantization on the performance of instruct models. We show that larger models show exceptional resilience to precision reduction and can maintain high accuracy even at 4-bit quantization.
  arXiv  Detail & Related papers  (2024-05-06T03:42:34Z)
• QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models [85.02796681773447]
  We propose a quantization-aware low-rank adaptation (QA-LoRA) algorithm. The motivation lies in the imbalanced degrees of freedom of quantization and adaptation. QA-LoRA is easily implemented with a few lines of code.
  arXiv  Detail & Related papers  (2023-09-26T07:22:23Z)
• Greener yet Powerful: Taming Large Code Generation Models with Quantization [47.734976584580224]
  Large pretrained deep learning models have substantially pushed the boundary of code generation. Despite their great power, the huge number of model parameters poses a significant threat to adapting them in a regular software development environment. Model compression is a promising approach to address these challenges.
  arXiv  Detail & Related papers  (2023-03-09T16:25:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.