On Security Weaknesses and Vulnerabilities in Deep Learning Systems

• URL: http://arxiv.org/abs/2406.08688v1
• Date: Wed, 12 Jun 2024 23:04:13 GMT
• Title: On Security Weaknesses and Vulnerabilities in Deep Learning Systems
• Authors: Zhongzheng Lai, Huaming Chen, Ruoxi Sun, Yu Zhang, Minhui Xue, Dong Yuan,
• Abstract summary: We specifically look into deep learning (DL) framework and perform the first systematic study of vulnerabilities in DL systems. We propose a two-stream data analysis framework to explore vulnerability patterns from various databases. We conducted a large-scale empirical study of 3,049 DL vulnerabilities to better understand the patterns of vulnerability and the challenges in fixing them.
• Score: 32.14068820256729
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The security guarantee of AI-enabled software systems (particularly using deep learning techniques as a functional core) is pivotal against the adversarial attacks exploiting software vulnerabilities. However, little attention has been paid to a systematic investigation of vulnerabilities in such systems. A common situation learned from the open source software community is that deep learning engineers frequently integrate off-the-shelf or open-source learning frameworks into their ecosystems. In this work, we specifically look into deep learning (DL) framework and perform the first systematic study of vulnerabilities in DL systems through a comprehensive analysis of identified vulnerabilities from Common Vulnerabilities and Exposures (CVE) and open-source DL tools, including TensorFlow, Caffe, OpenCV, Keras, and PyTorch. We propose a two-stream data analysis framework to explore vulnerability patterns from various databases. We investigate the unique DL frameworks and libraries development ecosystems that appear to be decentralized and fragmented. By revisiting the Common Weakness Enumeration (CWE) List, which provides the traditional software vulnerability related practices, we observed that it is more challenging to detect and fix the vulnerabilities throughout the DL systems lifecycle. Moreover, we conducted a large-scale empirical study of 3,049 DL vulnerabilities to better understand the patterns of vulnerability and the challenges in fixing them. We have released the full replication package at https://github.com/codelzz/Vulnerabilities4DLSystem. We anticipate that our study can advance the development of secure DL systems.

Related papers

• In-Context Experience Replay Facilitates Safety Red-Teaming of Text-to-Image Diffusion Models [97.82118821263825]
  Text-to-image (T2I) models have shown remarkable progress, but their potential to generate harmful content remains a critical concern in the ML community. We propose ICER, a novel red-teaming framework that generates interpretable and semantic meaningful problematic prompts. Our work provides crucial insights for developing more robust safety mechanisms in T2I systems.
  arXiv  Detail & Related papers  (2024-11-25T04:17:24Z)
• VulZoo: A Comprehensive Vulnerability Intelligence Dataset [12.229092589037808]
  VulZoo is a comprehensive vulnerability intelligence dataset that covers 17 popular vulnerability information sources. We make VulZoo publicly available and maintain it with incremental updates to facilitate future research.
  arXiv  Detail & Related papers  (2024-06-24T06:39:07Z)
• Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models [17.96542494363619]
  Large language models (LLMs) have demonstrated remarkable capabilities in comprehending complex contexts. In this paper, we conduct a study to investigate the capabilities of LLMs in both detecting and explaining vulnerabilities. Under specialized fine-tuning for vulnerability explanation, our LLMVulExp not only detects the types of vulnerabilities in the code but also analyzes the code context to generate the cause, location, and repair suggestions.
  arXiv  Detail & Related papers  (2024-06-14T04:01:25Z)
• Rethinking the Vulnerabilities of Face Recognition Systems:From a Practical Perspective [53.24281798458074]
  Face Recognition Systems (FRS) have increasingly integrated into critical applications, including surveillance and user authentication. Recent studies have revealed vulnerabilities in FRS to adversarial (e.g., adversarial patch attacks) and backdoor attacks (e.g., training data poisoning)
  arXiv  Detail & Related papers  (2024-05-21T13:34:23Z)
• Causative Insights into Open Source Software Security using Large Language Code Embeddings and Semantic Vulnerability Graph [3.623199159688412]
  Open Source Software (OSS) vulnerabilities can cause unauthorized access, data breaches, network disruptions, and privacy violations. Recent deep-learning techniques have shown great promise in identifying and localizing vulnerabilities in source code. Our study shows a 24% improvement in code repair capabilities compared to previous methods.
  arXiv  Detail & Related papers  (2024-01-13T10:33:22Z)
• HW-V2W-Map: Hardware Vulnerability to Weakness Mapping Framework for Root Cause Analysis with GPT-assisted Mitigation Suggestion [3.847218857469107]
  We presentHW-V2W-Map Framework, which is a Machine Learning (ML) framework focusing on hardware vulnerabilities and Internet of Things (IoT) security. The architecture that we have proposed incorporates an Ontology-driven Storytelling framework, which automates the process of updating the Ontology. Our proposed framework utilized Generative Pre-trained Transformer (GPT) Large Language Models (LLMs) to provide mitigation suggestions.
  arXiv  Detail & Related papers  (2023-12-21T02:14:41Z)
• ThreatKG: An AI-Powered System for Automated Open-Source Cyber Threat Intelligence Gathering and Management [65.0114141380651]
  ThreatKG is an automated system for OSCTI gathering and management. It efficiently collects a large number of OSCTI reports from multiple sources. It uses specialized AI-based techniques to extract high-quality knowledge about various threat entities.
  arXiv  Detail & Related papers  (2022-12-20T16:13:59Z)
• Security for Machine Learning-based Software Systems: a survey of threats, practices and challenges [0.76146285961466]
  How to securely develop the machine learning-based modern software systems (MLBSS) remains a big challenge. latent vulnerabilities and privacy issues exposed to external users and attackers will be largely neglected and hard to be identified. We consider that security for machine learning-based software systems may arise from inherent system defects or external adversarial attacks.
  arXiv  Detail & Related papers  (2022-01-12T23:20:25Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses [150.64470864162556]
  This work systematically categorizes and discusses a wide range of dataset vulnerabilities and exploits. In addition to describing various poisoning and backdoor threat models and the relationships among them, we develop their unified taxonomy.
  arXiv  Detail & Related papers  (2020-12-18T22:38:47Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.