Are Sparse Autoencoders Useful for Java Function Bug Detection?

• URL: http://arxiv.org/abs/2505.10375v3
• Date: Thu, 31 Jul 2025 19:17:07 GMT
• Title: Are Sparse Autoencoders Useful for Java Function Bug Detection?
• Authors: Rui Melo, Claudia Mamede, Andre Catarino, Rui Abreu, Henrique Lopes Cardoso,
• Abstract summary: Software vulnerabilities are a major source of security breaches.<n>Traditional methods for vulnerability detection are limited by high false positive rates, scalability issues, and reliance on manual effort.<n>Sparse Autoencoder offer a promising solution to this problem.
• Score: 5.119371135458389
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Software vulnerabilities such as buffer overflows and SQL injections are a major source of security breaches. Traditional methods for vulnerability detection remain essential but are limited by high false positive rates, scalability issues, and reliance on manual effort. These constraints have driven interest in AI-based approaches to automated vulnerability detection and secure code generation. While Large Language Models (LLMs) have opened new avenues for classification tasks, their complexity and opacity pose challenges for interpretability and deployment. Sparse Autoencoder offer a promising solution to this problem. We explore whether SAEs can serve as a lightweight, interpretable alternative for bug detection in Java functions. We evaluate the effectiveness of SAEs when applied to representations from GPT-2 Small and Gemma 2B, examining their capacity to highlight buggy behaviour without fine-tuning the underlying LLMs. We found that SAE-derived features enable bug detection with an F1 score of up to 89%, consistently outperforming fine-tuned transformer encoder baselines. Our work provides the first empirical evidence that SAEs can be used to detect software bugs directly from the internal representations of pretrained LLMs, without any fine-tuning or task-specific supervision. Code available at https://github.com/rufimelo99/SAE-Java-Bug-Detection

Related papers

• A Mixture of Linear Corrections Generates Secure Code [20.94236753015922]
  Large language models (LLMs) have become proficient at sophisticated code-generation tasks, yet remain ineffective at reliably detecting or avoiding code vulnerabilities.<n>We find that current LLMs encode precise internal representations that distinguish vulnerable from secure code.<n>We develop an inference-time steering technique that subtly modulates the model's token-generation probabilities through a mixture of corrections.
  arXiv  Detail & Related papers  (2025-07-13T06:27:33Z)
• Decompiling Smart Contracts with a Large Language Model [51.49197239479266]
  Despite Etherscan's 78,047,845 smart contracts deployed on (as of May 26, 2025), a mere 767,520 ( 1%) are open source.<n>This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode.<n>We introduce a pioneering decompilation pipeline that transforms bytecode into human-readable and semantically faithful Solidity code.
  arXiv  Detail & Related papers  (2025-06-24T13:42:59Z)
• SAVANT: Vulnerability Detection in Application Dependencies through Semantic-Guided Reachability Analysis [6.989158266868967]
  integration of open-source third-party library dependencies in Java development introduces significant security risks.<n>Savant combines semantic preprocessing with LLM-powered context analysis for accurate vulnerability detection.<n>Savant achieves 83.8% precision, 73.8% recall, 69.0% accuracy, and 78.5% F1-score, outperforming state-of-the-art SCA tools.
  arXiv  Detail & Related papers  (2025-06-21T19:48:13Z)
• Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders [50.52694757593443]
  Existing SAE training algorithms often lack rigorous mathematical guarantees and suffer from practical limitations.<n>We first propose a novel statistical framework for the feature recovery problem, which includes a new notion of feature identifiability.<n>We introduce a new SAE training algorithm based on bias adaptation'', a technique that adaptively adjusts neural network bias parameters to ensure appropriate activation sparsity.
  arXiv  Detail & Related papers  (2025-06-16T20:58:05Z)
• Training Language Models to Generate Quality Code with Program Analysis Feedback [66.0854002147103]
  Code generation with large language models (LLMs) is increasingly adopted in production but fails to ensure code quality.<n>We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code.
  arXiv  Detail & Related papers  (2025-05-28T17:57:47Z)
• Trace Gadgets: Minimizing Code Context for Machine Learning-Based Vulnerability Prediction [8.056137513320065]
  This work introduces Trace Gadgets, a novel code representation that minimizes code context by removing non-related code.<n>As input for ML models, Trace Gadgets provide a minimal but complete context, thereby improving the detection performance.<n>Our results show that state-of-the-art machine learning models perform best when using Trace Gadgets compared to previous code representations.
  arXiv  Detail & Related papers  (2025-04-18T13:13:39Z)
• Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs [60.881609323604685]
  Large Language Models (LLMs) accessed via black-box APIs introduce a trust challenge.<n>Users pay for services based on advertised model capabilities.<n> providers may covertly substitute the specified model with a cheaper, lower-quality alternative to reduce operational costs.<n>This lack of transparency undermines fairness, erodes trust, and complicates reliable benchmarking.
  arXiv  Detail & Related papers  (2025-04-07T03:57:41Z)
• Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories [8.583591493627276]
  We introduce JitVul, a vulnerability detection benchmark linking each function to its vulnerability-introducing and fixing commits.<n>We show that ReAct Agents, leveraging thought-action-observation and interprocedural context, perform better than LLMs in distinguishing vulnerable from benign code.
  arXiv  Detail & Related papers  (2025-03-05T15:22:24Z)
• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection [11.13802281700894]
  Static Application Security Testing (SAST) is usually utilized to scan source code for security vulnerabilities. Deep learning (DL)-based methods have demonstrated their potential in software vulnerability detection. This paper compares 15 diverse SAST tools with 12 popular or state-of-the-art open-source LLMs in detecting software vulnerabilities.
  arXiv  Detail & Related papers  (2024-07-23T07:21:14Z)
• Software Vulnerability and Functionality Assessment using LLMs [0.8057006406834466]
  We investigate whether Large Language Models (LLMs) can aid with code reviews. Our investigation focuses on two tasks that we argue are fundamental to good reviews.
  arXiv  Detail & Related papers  (2024-03-13T11:29:13Z)
• How Far Have We Gone in Vulnerability Detection Using Large Language Models [15.09461331135668]
  We introduce a comprehensive vulnerability benchmark VulBench. This benchmark aggregates high-quality data from a wide range of CTF challenges and real-world applications. We find that several LLMs outperform traditional deep learning approaches in vulnerability detection.
  arXiv  Detail & Related papers  (2023-11-21T08:20:39Z)
• Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities [12.82645410161464]
  We evaluate the effectiveness of 16 pre-trained Large Language Models on 5,000 code samples from five diverse security datasets. Overall, LLMs show modest effectiveness in detecting vulnerabilities, obtaining an average accuracy of 62.8% and F1 score of 0.71 across datasets. We find that advanced prompting strategies that involve step-by-step analysis significantly improve performance of LLMs on real-world datasets in terms of F1 score (by upto 0.18 on average)
  arXiv  Detail & Related papers  (2023-11-16T13:17:20Z)
• WSSOD: A New Pipeline for Weakly- and Semi-Supervised Object Detection [75.80075054706079]
  We propose a weakly- and semi-supervised object detection framework (WSSOD) An agent detector is first trained on a joint dataset and then used to predict pseudo bounding boxes on weakly-annotated images. The proposed framework demonstrates remarkable performance on PASCAL-VOC and MSCOCO benchmark, achieving a high performance comparable to those obtained in fully-supervised settings.
  arXiv  Detail & Related papers  (2021-05-21T11:58:50Z)
• Anomaly Detection Based on Selection and Weighting in Latent Space [73.01328671569759]
  We propose a novel selection-and-weighting-based anomaly detection framework called SWAD. Experiments on both benchmark and real-world datasets have shown the effectiveness and superiority of SWAD.
  arXiv  Detail & Related papers  (2021-03-08T10:56:38Z)
• Autosploit: A Fully Automated Framework for Evaluating the Exploitability of Security Vulnerabilities [47.748732208602355]
  Autosploit is an automated framework for evaluating the exploitability of vulnerabilities. It automatically tests the exploits on different configurations of the environment. It is able to identify the system properties that affect the ability to exploit a vulnerability in both noiseless and noisy environments.
  arXiv  Detail & Related papers  (2020-06-30T18:49:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.