An Empirical Study of Vulnerabilities in Python Packages and Their Detection

• URL: http://arxiv.org/abs/2509.04260v1
• Date: Thu, 04 Sep 2025 14:38:28 GMT
• Title: An Empirical Study of Vulnerabilities in Python Packages and Their Detection
• Authors: Haowei Quan, Junjie Wang, Xinzhe Li, Terry Yue Zhuo, Xiao Chen, Xiaoning Du,
• Abstract summary: This paper introduces PyVul, the first comprehensive benchmark suite of Python-package vulnerabilities.<n>PyVul includes 1,157 publicly reported, developer-verified vulnerabilities, each linked to its affected packages.<n>An LLM-assisted data cleansing method is incorporated to improve label accuracy, achieving 100% commit-level and 94% function-level accuracy.
• Score: 12.629138654621983
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In the rapidly evolving software development landscape, Python stands out for its simplicity, versatility, and extensive ecosystem. Python packages, as units of organization, reusability, and distribution, have become a pressing concern, highlighted by the considerable number of vulnerability reports. As a scripting language, Python often cooperates with other languages for performance or interoperability. This adds complexity to the vulnerabilities inherent to Python packages, and the effectiveness of current vulnerability detection tools remains underexplored. This paper addresses these gaps by introducing PyVul, the first comprehensive benchmark suite of Python-package vulnerabilities. PyVul includes 1,157 publicly reported, developer-verified vulnerabilities, each linked to its affected packages. To accommodate diverse detection techniques, it provides annotations at both commit and function levels. An LLM-assisted data cleansing method is incorporated to improve label accuracy, achieving 100% commit-level and 94% function-level accuracy, establishing PyVul as the most precise large-scale Python vulnerability benchmark. We further carry out a distribution analysis of PyVul, which demonstrates that vulnerabilities in Python packages involve multiple programming languages and exhibit a wide variety of types. Moreover, our analysis reveals that multi-lingual Python packages are potentially more susceptible to vulnerabilities. Evaluation of state-of-the-art detectors using this benchmark reveals a significant discrepancy between the capabilities of existing tools and the demands of effectively identifying real-world security issues in Python packages. Additionally, we conduct an empirical review of the top-ranked CWEs observed in Python packages, to diagnose the fine-grained limitations of current detection tools and highlight the necessity for future advancements in the field.

Related papers

• SAGA: Detecting Security Vulnerabilities Using Static Aspect Analysis [5.971445533193919]
  SAGA is an approach to detect and locate vulnerabilities in Python source code in a versatile way.<n>We have evaluated SAGA on a dataset of 108 vulnerabilities, obtaining 100% sensitivity and 99.15% specificity, with only one false positive, while outperforming four common security analysis tools.
  arXiv  Detail & Related papers  (2026-01-21T16:26:26Z)
• Why Authors and Maintainers Link (or Don't Link) Their PyPI Libraries to Code Repositories and Donation Platforms [83.16077040470975]
  Metadata of libraries on the Python Package Index (PyPI) plays a critical role in supporting the transparency, trust, and sustainability of open-source libraries.<n>This paper presents a large-scale empirical study combining two targeted surveys sent to 50,000 PyPI authors and maintainers.<n>We analyze more than 1,400 responses using large language model (LLM)-based topic modeling to uncover key motivations and barriers related to linking repositories and donation platforms.
  arXiv  Detail & Related papers  (2026-01-21T16:13:57Z)
• Analyzing the Availability of E-Mail Addresses for PyPI Libraries [89.21869606965578]
  81.6% of libraries include at least one valid e-mail address, with PyPI serving as the primary source.<n>We identify over 698,000 invalid entries, primarily due to missing fields.
  arXiv  Detail & Related papers  (2026-01-20T14:54:58Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• PyPitfall: Dependency Chaos and Software Supply Chain Vulnerabilities in Python [1.2644387713029346]
  This paper introduces PyPitfall, a quantitative analysis of vulnerable dependencies across the PyPI ecosystem.<n>We analyzed the dependency structures of 378,573 PyPI packages and identified 4,655 packages that explicitly require at least one known-vulnerable version.<n>We aim to raise awareness of Python software supply chain security by characterizing the ecosystem-wide dependency landscape.
  arXiv  Detail & Related papers  (2025-07-24T03:58:18Z)
• PyPulse: A Python Library for Biosignal Imputation [58.35269251730328]
  We introduce PyPulse, a Python package for imputation of biosignals in both clinical and wearable sensor settings.<n>PyPulse's framework provides a modular and extendable framework with high ease-of-use for a broad userbase, including non-machine-learning bioresearchers.<n>We released PyPulse under the MIT License on Github and PyPI.
  arXiv  Detail & Related papers  (2024-12-09T11:00:55Z)
• A Machine Learning-Based Approach For Detecting Malicious PyPI Packages [4.311626046942916]
  In modern software development, the use of external libraries and packages is increasingly prevalent.<n>This reliance on reusing code introduces serious risks for deployed software in the form of malicious packages.<n>We propose a data-driven approach that uses machine learning and static analysis to examine the package's metadata, code, files, and textual characteristics.
  arXiv  Detail & Related papers  (2024-12-06T18:49:06Z)
• An Empirical Study of Vulnerability Handling Times in CPython [0.2538209532048867]
  The paper examines the handling times of software vulnerabilities in CPython.<n>The paper contributes to the recent effort to better understand security of the Python ecosystem.
  arXiv  Detail & Related papers  (2024-11-01T08:46:14Z)
• An Empirical Study on Package-Level Deprecation in Python Ecosystem [6.0347124337922144]
  Python, a widely adopted programming language, is renowned for its extensive and diverse third-party package ecosystem. A significant number of OSS packages within the Python ecosystem are in poor maintenance, leading to potential risks in functionality and security. This paper investigates the current practices of announcing, receiving, and handling package-level deprecation in the Python ecosystem.
  arXiv  Detail & Related papers  (2024-08-19T18:08:21Z)
• PyGOD: A Python Library for Graph Outlier Detection [56.33769221859135]
  PyGOD is an open-source library for detecting outliers in graph data. It supports a wide array of leading graph-based methods for outlier detection. PyGOD is released under a BSD 2-Clause license at https://pygod.org and at the Python Package Index (PyPI)
  arXiv  Detail & Related papers  (2022-04-26T06:15:21Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• Scikit-dimension: a Python package for intrinsic dimension estimation [58.8599521537]
  This technical note introduces textttscikit-dimension, an open-source Python package for intrinsic dimension estimation. textttscikit-dimension package provides a uniform implementation of most of the known ID estimators based on scikit-learn application programming interface. We briefly describe the package and demonstrate its use in a large-scale (more than 500 datasets) benchmarking of methods for ID estimation in real-life and synthetic data.
  arXiv  Detail & Related papers  (2021-09-06T16:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.