LLM-Enabled Open-Source Systems in the Wild: An Empirical Study of Vulnerabilities in GitHub Security Advisories

• URL: http://arxiv.org/abs/2604.04288v1
• Date: Sun, 05 Apr 2026 22:07:31 GMT
• Title: LLM-Enabled Open-Source Systems in the Wild: An Empirical Study of Vulnerabilities in GitHub Security Advisories
• Abstract summary: We analyze 295 GitHub Security Advisories published between January 2025 and January 2026.<n>We find no evidence of new implementation-level weakness classes specific to LLM systems.<n>Most advisories map to established CWEs, particularly injection and deization weaknesses.
• Score: 3.1744585256350053
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly embedded in open-source software (OSS) ecosystems, creating complex interactions among natural language prompts, probabilistic model outputs, and execution-capable components. However, it remains unclear whether traditional vulnerability disclosure frameworks adequately capture these model-mediated risks. To investigate this, we analyze 295 GitHub Security Advisories published between January 2025 and January 2026 that reference LLM-related components, and we manually annotate a sample of 100 advisories using the OWASP Top 10 for LLM Applications 2025. We find no evidence of new implementation-level weakness classes specific to LLM systems. Most advisories map to established CWEs, particularly injection and deserialization weaknesses. At the same time, the OWASP-based analysis reveals recurring architectural risk patterns, especially Supply Chain, Excessive Agency, and Prompt Injection, which often co-occur across multiple stages of execution. These results suggest that existing advisory metadata captures code-level defects but underrepresents model-mediated exposure. We conclude that combining the CWE and OWASP perspectives provides a more complete and necessary view of vulnerabilities in LLM-integrated systems.

Related papers

• Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability [65.56750741053266]
  Large language models (LLMs) have transformed misinformation from a content-centric problem into a broader ecosystem-level security challenge.<n>In this paper, we introduce a role-layer framework to unify these risks and defenses.<n>The role characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification.
  arXiv  Detail & Related papers  (2026-07-11T17:04:17Z)
• Towards Demystifying and Repairing LLM-in-the-Loop Vulnerabilities [22.313168715963315]
  Large Language Models (LLMs) have been actively integrated into modern software systems as critical components.<n>LLMs-in-the-loop vulnerabilities, where vulnerabilities are introduced by LLMs and their dependent downstream components, such as frameworks, introduce new risks.<n>These vulnerabilities are more challenging to precisely fix, especially for those involving prompt injections where the Pass@1 rate is only 28.57%.
  arXiv  Detail & Related papers  (2026-05-27T08:11:37Z)
• An Empirical Evaluation of LLM-Based Approaches for Code Vulnerability Detection: RAG, SFT, and Dual-Agent Systems [1.5216960763930782]
  The rapid advancement of Large Language Models (LLMs) presents new opportunities for automated software vulnerability detection.<n>This paper presents a comparative study on the effectiveness of LLM-based techniques for detecting software vulnerabilities.
  arXiv  Detail & Related papers  (2026-01-01T08:05:51Z)
• Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography [77.44136793431893]
  We propose a novel jailbreak paradigm that introduces dual steganography to covertly embed malicious queries into benign-looking images.<n>Our Odysseus successfully jailbreaks several pioneering and realistic MLLM-integrated systems, achieving up to 99% attack success rate.
  arXiv  Detail & Related papers  (2025-12-23T08:53:36Z)
• Automated Vulnerability Validation and Verification: A Large Language Model Approach [7.482522010482827]
  This paper introduces an end-to-end multi-step pipeline leveraging generative AI, specifically large language models (LLMs)<n>Our approach extracts information from CVE disclosures in the National Vulnerability Database.<n>It augments it with external public knowledge (e.g., threat advisories, code snippets) using Retrieval-Augmented Generation (RAG)<n>The pipeline iteratively refines generated artifacts, validates attack success with test cases, and supports complex multi-container setups.
  arXiv  Detail & Related papers  (2025-09-28T19:16:12Z)
• An Empirical Study of Vulnerable Package Dependencies in LLM Repositories [14.817045028745563]
  Large language models (LLMs) rely on external code dependencies from package management systems.<n>Vulnerabilities in dependencies can expose LLMs to security risks.<n>Half of the vulnerabilities in the LLM ecosystem remain undisclosed for more than 56.2 months, significantly longer than those in the Python ecosystem.
  arXiv  Detail & Related papers  (2025-08-29T08:38:58Z)
• Understanding the Supply Chain and Risks of Large Language Model Applications [25.571274158366563]
  We introduce the first comprehensive dataset for analyzing and benchmarking Large Language Models (LLMs) supply chain security.<n>We collect 3,859 real-world LLM applications and perform interdependency analysis, identifying 109,211 models, 2,474 datasets, and 9,862 libraries.<n>Our findings reveal deeply nested dependencies in LLM applications and significant vulnerabilities across the supply chain, underscoring the need for comprehensive security analysis.
  arXiv  Detail & Related papers  (2025-07-24T05:30:54Z)
• Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Study [7.5198516000202105]
  Large language models (LLMs) are increasingly deployed through open-source and commercial frameworks.<n>As LLM deployments become prevalent, particularly in industry, ensuring their secure and reliable operation has become a critical issue.<n>Insecure defaults and misconfigurations often expose LLM services to the public internet, posing serious security and system engineering risks.
  arXiv  Detail & Related papers  (2025-05-05T09:30:19Z)
• Reasoning with LLMs for Zero-Shot Vulnerability Detection [0.9208007322096533]
  We present textbfVulnSage, a comprehensive evaluation framework and a curated dataset from diverse, large-scale open-source system software projects.<n>The framework supports multi-granular analysis across function, file, and inter-function levels.<n>It employs four diverse zero-shot prompt strategies: Baseline, Chain-of-context, Think, and Think & verify.
  arXiv  Detail & Related papers  (2025-03-22T23:59:17Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• OR-Bench: An Over-Refusal Benchmark for Large Language Models [65.34666117785179]
  Large Language Models (LLMs) require careful safety alignment to prevent malicious outputs.<n>This study proposes a novel method for automatically generating large-scale over-refusal datasets.<n>We introduce OR-Bench, the first large-scale over-refusal benchmark.
  arXiv  Detail & Related papers  (2024-05-31T15:44:33Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.