Understanding the Supply Chain and Risks of Large Language Model Applications

• URL: http://arxiv.org/abs/2507.18105v1
• Date: Thu, 24 Jul 2025 05:30:54 GMT
• Title: Understanding the Supply Chain and Risks of Large Language Model Applications
• Authors: Yujie Ma, Lili Quan, Xiaofei Xie, Qiang Hu, Jiongchi Yu, Yao Zhang, Sen Chen,
• Abstract summary: We introduce the first comprehensive dataset for analyzing and benchmarking Large Language Models (LLMs) supply chain security.<n>We collect 3,859 real-world LLM applications and perform interdependency analysis, identifying 109,211 models, 2,474 datasets, and 9,862 libraries.<n>Our findings reveal deeply nested dependencies in LLM applications and significant vulnerabilities across the supply chain, underscoring the need for comprehensive security analysis.
• Score: 25.571274158366563
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rise of Large Language Models (LLMs) has led to the widespread deployment of LLM-based systems across diverse domains. As these systems proliferate, understanding the risks associated with their complex supply chains is increasingly important. LLM-based systems are not standalone as they rely on interconnected supply chains involving pretrained models, third-party libraries, datasets, and infrastructure. Yet, most risk assessments narrowly focus on model or data level, overlooking broader supply chain vulnerabilities. While recent studies have begun to address LLM supply chain risks, there remains a lack of benchmarks for systematic research. To address this gap, we introduce the first comprehensive dataset for analyzing and benchmarking LLM supply chain security. We collect 3,859 real-world LLM applications and perform interdependency analysis, identifying 109,211 models, 2,474 datasets, and 9,862 libraries. We extract model fine-tuning paths, dataset reuse, and library reliance, mapping the ecosystem's structure. To evaluate security, we gather 1,555 risk-related issues-50 for applications, 325 for models, 18 for datasets, and 1,229 for libraries from public vulnerability databases. Using this dataset, we empirically analyze component dependencies and risks. Our findings reveal deeply nested dependencies in LLM applications and significant vulnerabilities across the supply chain, underscoring the need for comprehensive security analysis. We conclude with practical recommendations to guide researchers and developers toward safer, more trustworthy LLM-enabled systems.

Related papers

• Drop the Golden Apples: Identifying Third-Party Reuse by DB-Less Software Composition Analysis [11.193453132177222]
  Third-party libraries (TPLs) in modern software development introduce significant security and compliance risks.<n>We propose the first framework of DB-Less Software Composition Analysis (SCA) to get rid of the traditional heavy database.<n>Our experiments on two typical scenarios, native library identification for Android and copy-based TPL reuse for C/C++, have demonstrated the favorable future for implementing database-less strategies in SCA.
  arXiv  Detail & Related papers  (2025-03-28T16:25:24Z)
• Datenschutzkonformer LLM-Einsatz: Eine Open-Source-Referenzarchitektur [0.10713888959520207]
  We present a reference architecture for developing closed, LLM-based systems using open-source technologies.<n>The architecture provides a flexible and transparent solution that meets strict data privacy and security requirements.
  arXiv  Detail & Related papers  (2025-03-01T14:51:07Z)
• LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights [12.424610893030353]
  Large Language Models (LLMs) are emerging as transformative tools for software vulnerability detection.<n>This paper provides a detailed survey of LLMs in vulnerability detection.<n>We address challenges such as cross-language vulnerability detection, multimodal data integration, and repository-level analysis.
  arXiv  Detail & Related papers  (2025-02-10T21:33:38Z)
• Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents [67.07177243654485]
  This survey collects and analyzes the different threats faced by large language models-based agents. We identify six key features of LLM-based agents, based on which we summarize the current research progress. We select four representative agents as case studies to analyze the risks they may face in practical use.
  arXiv  Detail & Related papers  (2024-11-14T15:40:04Z)
• Lifting the Veil on Composition, Risks, and Mitigations of the Large Language Model Supply Chain [6.478930807409979]
  Large language models (LLMs) have sparked significant impact with regard to both intelligence and productivity.<n>We develop a structured taxonomy encompassing risk types, risky actions, and corresponding mitigations across different stakeholders.
  arXiv  Detail & Related papers  (2024-10-28T17:02:12Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety [27.843894102000608]
  We conduct a first systematic review of open datasets for evaluating and improving large language models (LLMs) safety.<n>We highlight trends, such as a trend towards fully synthetic datasets, as well as gaps in dataset coverage, such as a clear lack of non-English and naturalistic datasets.<n>Our contributions are based on SafetyPrompts.com, a living catalogue of open datasets for LLM safety.
  arXiv  Detail & Related papers  (2024-04-08T10:57:25Z)
• A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems [47.18371401090435]
  We analyze the security of Large Language Model (LLM) systems, instead of focusing on the individual LLMs. We propose a multi-layer and multi-step approach and apply it to the state-of-art OpenAI GPT4. We found that although the OpenAI GPT4 has designed numerous safety constraints to improve its safety features, these safety constraints are still vulnerable to attackers.
  arXiv  Detail & Related papers  (2024-02-28T19:00:12Z)
• Self-Retrieval: End-to-End Information Retrieval with One Large Language Model [97.71181484082663]
  We introduce Self-Retrieval, a novel end-to-end LLM-driven information retrieval architecture. Self-Retrieval internalizes the retrieval corpus through self-supervised learning, transforms the retrieval process into sequential passage generation, and performs relevance assessment for reranking.
  arXiv  Detail & Related papers  (2024-02-23T18:45:35Z)
• Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs [59.596335292426105]
  This paper collects the first open-source dataset to evaluate safeguards in large language models. We train several BERT-like classifiers to achieve results comparable with GPT-4 on automatic safety evaluation.
  arXiv  Detail & Related papers  (2023-08-25T14:02:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.