Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection

• URL: http://arxiv.org/abs/2605.29901v1
• Date: Thu, 28 May 2026 13:23:13 GMT
• Title: Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection
• Abstract summary: We analyze the internal computations of a neural network to understand its reasoning process.<n>Using Circuit Tracer on Gemma-2-2b, we trace the computational pathways activated when the model classifies 472 C/C++ code samples as vulnerable or safe.
• Score: 1.9298849497208217
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) can detect software vulnerabilities, but how do they actually identify vulnerable code? We address this question using mechanistic interpretability; analyzing the internal computations of a neural network to understand its reasoning process.Using Circuit Tracer on Gemma-2-2b, we trace the computational pathways activated when the model classifies 472 C/C++ code samples as vulnerable or safe. Our analysis reveals a surprising finding: the model primarily relies on safety detectors, attention heads that recognize safe coding patterns, rather than directly detecting vulnerability signatures. When these safety detectors fail to activate, the model classifies code as vulnerable. We identify the critical neural components: specific attention heads in early layers (L5, L7) that focus on safety patterns, and Multilayer Perceptron (MLP) neurons in Layer 7 that encode vulnerability-related features. Ablation experiments confirm their causal role; removing Layer 11 drops vulnerability detection accuracy from 100% to 6%, while ablating just 20 neurons in Layer 7 reduces it by 50%.Our findings show that LLM vulnerability detection uses sparse, interpretable circuits (only 16% of model capacity), enabling circuit-level explanations for security predictions and targeted improvements to detection systems.

Related papers

• ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments [12.021872048536766]
  Large language models are increasingly deployed for security-sensitive tasks such as vulnerability detection and code review.<n>Their reliance on natural-language context embedded in source code exposes a previously underexplored attack surface: adversarial comments.<n>We present ALIBI, an automated adaptive black-box attack framework that generates and iteratively refines adversarial comments.
  arXiv  Detail & Related papers  (2026-07-27T18:13:28Z)
• Which Neurons Detect Malicious Code? A Probing Study of LLM Security Knowledge [1.6986037642844662]
  Large language models (LLMs) have become increasingly capable of understanding and generating source code.<n>We investigated where the malware detection behavior is encoded inside LLMs Feed Forward Network (FFN) neurons.<n>This aims to identify the most important neurons in detecting malicious code.
  arXiv  Detail & Related papers  (2026-07-11T09:21:01Z)
• EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy [66.59724477993339]
  Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to backdoor attacks.<n>We propose EntropyScan, a lightweight and trigger-agnostic method for model-level backdoor detection in LVLMs.<n>Our code will be publicly available soon.
  arXiv  Detail & Related papers  (2026-05-15T08:01:32Z)
• ShieldNet: Network-Level Guardrails against Emerging Supply-Chain Injections in Agentic Systems [56.613157564882925]
  Malicious behaviors are embedded in seemingly benign tools, silently hijacking agent execution, leaking sensitive data, or triggering unauthorized actions.<n>Despite their growing impact, there is currently no comprehensive benchmark for evaluating such threats.<n>We propose ShieldNet, a network-level guardrail framework that detects supply-chain poisoning by observing real network interactions.
  arXiv  Detail & Related papers  (2026-04-06T05:15:00Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• Defending against Indirect Prompt Injection by Instruction Detection [109.30156975159561]
  InstructDetector is a novel detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks.<n>InstructDetector achieves a detection accuracy of 99.60% in the in-domain setting and 96.90% in the out-of-domain setting, and reduces the attack success rate to just 0.03% on the BIPIA benchmark.
  arXiv  Detail & Related papers  (2025-05-08T13:04:45Z)
• Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons [57.07507194465299]
  Large language models (LLMs) excel in various capabilities but pose safety risks such as generating harmful content and misinformation, even after safety alignment.<n>We focus on identifying and analyzing safety neurons within LLMs that are responsible for safety behaviors.<n>We propose inference-time activation contrasting to locate these neurons and dynamic activation patching to evaluate their causal effects on model safety.
  arXiv  Detail & Related papers  (2024-06-20T09:35:22Z)
• On the Effectiveness of Function-Level Vulnerability Detectors for Inter-Procedural Vulnerabilities [28.57872406228216]
  We propose a tool dubbed VulTrigger for identifying vulnerability-triggering statements across functions. Experimental results show that VulTrigger can effectively identify vulnerability-triggering statements and inter-procedural vulnerabilities. Our findings include: (i) inter-procedural vulnerabilities are prevalent with an average of 2.8 inter-procedural layers; and (ii) function-level vulnerability detectors are much less effective in detecting to-be-patched functions of inter-procedural vulnerabilities.
  arXiv  Detail & Related papers  (2024-01-18T07:32:11Z)
• SliceLocator: Locating Vulnerable Statements with Graph-based Detectors [33.395068754566935]
  SliceLocator identifies the most relevant taint flow by selecting the highest-weighted flow path from all potential vulnerability-triggering statements.<n>We demonstrate that SliceLocator consistently performs well on four state-of-the-art GNN-based vulnerability detectors.
  arXiv  Detail & Related papers  (2024-01-05T10:15:04Z)
• Deep-Learning-based Vulnerability Detection in Binary Executables [0.0]
  We present a supervised deep learning approach using recurrent neural networks for the application of vulnerability detection based on binary executables. A dataset with 50,651 samples of vulnerable code in the form of a standardized LLVM Intermediate Representation is used. A binary classification was established for detecting the presence of an arbitrary vulnerability, and a multi-class model was trained for the identification of the exact vulnerability.
  arXiv  Detail & Related papers  (2022-11-25T10:33:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.