VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection

• URL: http://arxiv.org/abs/2509.11523v1
• Date: Mon, 15 Sep 2025 02:25:38 GMT
• Title: VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection
• Abstract summary: VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
• Score: 55.957275374847484
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The application of language models to project-level vulnerability detection remains challenging, owing to the dual requirement of accurately localizing security-sensitive code and correctly correlating and reasoning over complex program context. We present VulAgent, a multi-agent vulnerability detection framework based on hypothesis validation. Our design is inspired by how human auditors review code: when noticing a sensitive operation, they form a hypothesis about a possible vulnerability, consider potential trigger paths, and then verify the hypothesis against the surrounding context. VulAgent implements a semantics-sensitive, multi-view detection pipeline: specialized agents, each aligned to a specific analysis perspective (e.g., memory, authorization), collaboratively surface and precisely localize sensitive code sites with higher coverage. Building on this, VulAgent adopts a hypothesis-validation paradigm: for each vulnerability report, it builds hypothesis conditions and a trigger path, steering the LLM to target the relevant program context and defensive checks during verification, which reduces false positives. On average across the two datasets, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450% (246% on average), and reduces the false positive rate by about 36% compared with state-of-the-art LLM-based baselines.

Related papers

• Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale [54.83971397981572]
  Language-model agents increasingly operate over complete software repositories.<n> cybersecurity evaluations measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant code.<n>We study vulnerability localization: given a weakness class and an unfamiliar repository, identify the implementation files associated with that weakness.
  arXiv  Detail & Related papers  (2026-09-14T17:44:51Z)
• ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments [12.021872048536766]
  Large language models are increasingly deployed for security-sensitive tasks such as vulnerability detection and code review.<n>Their reliance on natural-language context embedded in source code exposes a previously underexplored attack surface: adversarial comments.<n>We present ALIBI, an automated adaptive black-box attack framework that generates and iteratively refines adversarial comments.
  arXiv  Detail & Related papers  (2026-07-27T18:13:28Z)
• DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection [16.007199928732614]
  We present DREA, a hypothesis-driven framework for repository-level vulnerability detection.<n>Goal-directed context acquisition is the primary source of detection improvement in this design.<n>To support evaluation, we construct RepoPairBench, a repository-grounded benchmark of validated Python vulnerability-fix pairs from real-world projects.
  arXiv  Detail & Related papers  (2026-07-15T04:49:05Z)
• Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing [8.293465530619246]
  We propose a two-stage decoupled evaluation framework that separates exploit execution from reconnaissance.<n>We empirically evaluate five open-source penetration-testing agents, covering multiagent, monolithic, and graph-driven architectures.
  arXiv  Detail & Related papers  (2026-06-24T02:51:58Z)
• VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems [79.51005192758262]
  Large language model-driven multi-agent systems excel at complex tasks.<n>However, unreliable agents remain a key bottleneck to system-level reliability.<n>We propose a hypothesis verification framework for agent failure attribution.
  arXiv  Detail & Related papers  (2026-05-17T14:09:35Z)
• Measuring and Exploiting Confirmation Bias in LLM-Assisted Security Code Review [6.417595678110472]
  We study whether confirmation bias affects LLM-based vulnerability detection, and whether this failure mode can be exploited in software supply-chain attacks.<n>Study 1 quantifies confirmation bias through controlled experiments on 250 CVE vulnerability/patch pairs evaluated across four state-of-the-art models under five framing conditions for the review prompt.<n>Study 2 evaluates exploitability in practice mimicking adversarial pull requests that reintroduce known vulnerabilities while framed as security improvements or urgent functionality fixes via their pull request metadata.
  arXiv  Detail & Related papers  (2026-03-19T10:40:27Z)
• VulnAgent-X: A Layered Agentic Framework for Repository-Level Vulnerability Detection [9.51089779418208]
  VulnAgentX is a layered agentic framework integrat- ing lightweight risk screening, bounded context expansion, specialised analysis agents, selective dynamic verification, and evidence fusion into a unified pipeline.<n>Treating vulnerabil- ity detection as a staged, evidence-driven auditing process improves de- tection quality, reduces false positives, and produces interpretable re-sults for repository-level software security analysis.
  arXiv  Detail & Related papers  (2026-03-11T04:57:17Z)
• Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generation [36.950993500170014]
  We present DrillAgent, an agentic framework that reformulates PoV generation as an iterative hypothesis-verification-refinement process.<n>We evaluate DrillAgent on SEC-bench, a large-scale benchmark of real-world C/C++ vulnerabilities.
  arXiv  Detail & Related papers  (2026-02-14T03:17:27Z)
• MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution [28.062506040151153]
  Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to two key limitations.<n>The heterogeneity of vulnerability patterns undermines the effectiveness of a single unified model, and manual prompt engineering for massive weakness categories is unscalable.<n>We propose textbfMulVul, a retrieval-augmented multi-agent framework for precise and broad-coverage vulnerability detection.
  arXiv  Detail & Related papers  (2026-01-26T12:43:10Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection [76.91230292971115]
  Large language model (LLM)-based multi-agent systems (MAS) have shown strong capabilities in solving complex tasks.<n>XG-Guard is an explainable and fine-grained safeguarding framework for detecting malicious agents in MAS.
  arXiv  Detail & Related papers  (2025-12-21T13:46:36Z)
• SecureAgentBench: Benchmarking Secure Code Generation under Realistic Vulnerability Scenarios [17.276786247873613]
  SecureAgentBench is a benchmark of 105 coding tasks designed to rigorously evaluate code agents' capabilities in secure code generation.<n>Results show that (i) current agents struggle to produce secure code, as even the best-performing one, SWE-agent supported by DeepSeek-V3.1, achieves merely 15.2% correct-and-secure solutions.
  arXiv  Detail & Related papers  (2025-09-26T09:18:57Z)
• Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks [0.5218155982819203]
  Large Language Models (LLMs) are increasingly used as code assistants.<n>This study examines a more direct threat: open-source LLMs generating vulnerable code when prompted.
  arXiv  Detail & Related papers  (2025-07-14T08:36:26Z)
• Lie Detector: Unified Backdoor Detection via Cross-Examination Framework [68.45399098884364]
  We propose a unified backdoor detection framework in the semi-honest setting.<n>Our method achieves superior detection performance, improving accuracy by 5.4%, 1.6%, and 11.9% over SoTA baselines.<n> Notably, it is the first to effectively detect backdoors in multimodal large language models.
  arXiv  Detail & Related papers  (2025-03-21T06:12:06Z)
• Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories [8.583591493627276]
  We introduce JitVul, a vulnerability detection benchmark linking each function to its vulnerability-introducing and fixing commits.<n>We show that ReAct Agents, leveraging thought-action-observation and interprocedural context, perform better than LLMs in distinguishing vulnerable from benign code.
  arXiv  Detail & Related papers  (2025-03-05T15:22:24Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)
• Malicious Agent Detection for Robust Multi-Agent Collaborative Perception [52.261231738242266]
  Multi-agent collaborative (MAC) perception is more vulnerable to adversarial attacks than single-agent perception. We propose Malicious Agent Detection (MADE), a reactive defense specific to MAC perception. We conduct comprehensive evaluations on a benchmark 3D dataset V2X-sim and a real-road dataset DAIR-V2X.
  arXiv  Detail & Related papers  (2023-10-18T11:36:42Z)
• Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives [8.524720028421447]
  This paper provides a systematic analysis of the opportunities, challenges, and potential solutions of harnessing Large Language Models (LLMs) such as GPT-4. generating more answers with higher randomness largely boosts the likelihood of producing a correct answer but inevitably leads to a higher number of false positives. We propose an adversarial framework dubbed GPTLens that breaks the conventional one-stage detection into two synergistic stages $-$ generation and discrimination.
  arXiv  Detail & Related papers  (2023-10-02T12:37:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.