Security Knowledge-Guided Fuzzing of Deep Learning Libraries

• URL: http://arxiv.org/abs/2306.03269v2
• Date: Sun, 24 Dec 2023 23:50:31 GMT
• Title: Security Knowledge-Guided Fuzzing of Deep Learning Libraries
• Authors: Nima Shiri Harzevili, Mohammad Mahdi Mohajer, Moshi Wei, Hung Viet Pham, Song Wang
• Abstract summary: We propose a novel Deep Learning fuzzer named Orion. Orion combines guided test input generation and corner case test input generation based on a set of fuzzing rules.
• Score: 9.930638894226004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, many Deep Learning fuzzers have been proposed for testing of DL libraries. However, they either perform unguided input generation (e.g., not considering the relationship between API arguments when generating inputs) or only support a limited set of corner case test inputs. Furthermore, a substantial number of developer APIs crucial for library development remain untested, as they are typically not well-documented and lack clear usage guidelines. To fill this gap, we propose a novel fuzzer named Orion, which combines guided test input generation and corner case test input generation based on a set of fuzzing rules constructed from historical data that is known to trigger vulnerabilities in the implementation of DL APIs. To extract the fuzzing rules, we first conduct an empirical study regarding the root cause analysis of 376 vulnerabilities in two of the most popular DL libraries, i.e., PyTorch and TensorFlow. We then construct the rules based on the root causes of the historical vulnerabilities. Our evaluation shows that Orion reports 135 vulnerabilities on the latest releases of TensorFlow and PyTorch, 76 of which were confirmed by the library developers. Among the 76 confirmed vulnerabilities, 69 are previously unknown, and 7 have already been fixed. The rest are awaiting further confirmation. Regarding end-user APIs, Orion was able to detect 31.8% and 90% more vulnerabilities on TensorFlow and PyTorch, respectively, compared to the state-of-the-art conventional fuzzer, i.e., DeepRel. When compared to the state-of-the-art LLM-based DL fuzzer, AtlasFuzz, Orion detected 13.63% more vulnerabilities on TensorFlow and 18.42% more vulnerabilities on PyTorch. Regarding developer APIs, Orion stands out by detecting 117% more vulnerabilities on TensorFlow and 100% more vulnerabilities on PyTorch compared to the most relevant fuzzer designed for developer APIs, such as FreeFuzz.

Related papers

• FuzzTheREST: An Intelligent Automated Black-box RESTful API Fuzzer [0.0]
  This work introduces a black-box API of fuzzy testing tool that employs Reinforcement Learning (RL) for vulnerability detection. The tool found a total of six unique vulnerabilities and achieved 55% code coverage.
  arXiv  Detail & Related papers  (2024-07-19T14:43:35Z)
• Mining REST APIs for Potential Mass Assignment Vulnerabilities [1.0377683220196872]
  We propose a lightweight approach to mine the REST API specifications and identify operations and attributes that are prone to mass assignment. We conducted a preliminary study on 100 APIs and found 25 prone to this vulnerability. We confirmed nine real vulnerable operations in six APIs.
  arXiv  Detail & Related papers  (2024-05-02T09:19:32Z)
• Model X-ray:Detect Backdoored Models via Decision Boundary [66.41173675107886]
  Deep neural networks (DNNs) have revolutionized various industries, leading to the rise of Machine Learning as a Service (ML) DNNs are susceptible to backdoor attacks, which pose significant risks to their applications. We propose Model X-ray, a novel backdoor detection approach for ML through the analysis of decision boundaries.
  arXiv  Detail & Related papers  (2024-02-27T12:42:07Z)
• Benchmarking Deep Learning Fuzzers [11.118370064698869]
  We run three state-of-the-art DL fuzzers, FreeFuzz, DeepRel, and DocTer, on the benchmark by following their instructions. We find that these fuzzers are unable to detect many real bugs collected in our benchmark dataset. Our systematic analysis further identifies four major, broad, and common factors that affect these fuzzers' ability to detect real bugs.
  arXiv  Detail & Related papers  (2023-10-10T18:09:16Z)
• SyzTrust: State-aware Fuzzing on Trusted OS Designed for IoT Devices [67.65883495888258]
  We present SyzTrust, the first state-aware fuzzing framework for vetting the security of resource-limited Trusted OSes. SyzTrust adopts a hardware-assisted framework to enable fuzzing Trusted OSes directly on IoT devices. We evaluate SyzTrust on Trusted OSes from three major vendors: Samsung, Tsinglink Cloud, and Ali Cloud.
  arXiv  Detail & Related papers  (2023-09-26T08:11:38Z)
• On the Security Blind Spots of Software Composition Analysis [46.1389163921338]
  We present a novel approach to detect vulnerable clones in the Maven repository. We retrieve over 53k potential vulnerable clones from Maven Central. We detect 727 confirmed vulnerable clones and synthesize a testable proof-of-vulnerability project for each of those.
  arXiv  Detail & Related papers  (2023-06-08T20:14:46Z)
• EDEFuzz: A Web API Fuzzer for Excessive Data Exposures [3.5061201620029885]
  Excessive Data Exposure (EDE) was the third most significant API vulnerability of 2019. There are few automated tools -- either in research or industry -- to effectively find and remediate such issues. We build the first fuzzing tool -- that we call EDEFuzz -- to systematically detect EDEs.
  arXiv  Detail & Related papers  (2023-01-23T04:05:08Z)
• REaaS: Enabling Adversarially Robust Downstream Classifiers via Robust Encoder as a Service [67.0982378001551]
  We show how a service provider pre-trains an encoder and then deploys it as a cloud service API. A client queries the cloud service API to obtain feature vectors for its training/testing inputs. We show that the cloud service only needs to provide two APIs to enable a client to certify the robustness of its downstream classifier.
  arXiv  Detail & Related papers  (2023-01-07T17:40:11Z)
• Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection [69.59980270078067]
  We explore the untargeted backdoor watermarking scheme, where the abnormal model behaviors are not deterministic. We also discuss how to use the proposed untargeted backdoor watermark for dataset ownership verification.
  arXiv  Detail & Related papers  (2022-09-27T12:56:56Z)
• VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Python [8.810543294798485]
  VUDENC is a deep learning-based vulnerability detection tool. It learns features of vulnerable code from a large and real-world Python corpus. VUDENC achieves a recall of 78%-87%, a precision of 82%-96%, and an F1 score of 80%-90%.
  arXiv  Detail & Related papers  (2022-01-20T20:29:22Z)
• Detection as Regression: Certified Object Detection by Median Smoothing [50.89591634725045]
  This work is motivated by recent progress on certified classification by randomized smoothing. We obtain the first model-agnostic, training-free, and certified defense for object detection against $ell$-bounded attacks.
  arXiv  Detail & Related papers  (2020-07-07T18:40:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.