CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph

• URL: http://arxiv.org/abs/2411.11532v3
• Date: Fri, 20 Dec 2024 11:25:59 GMT
• Title: CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph
• Authors: Hanxiang Xu, Wei Ma, Ting Zhou, Yanjie Zhao, Kai Chen, Qiang Hu, Yang Liu, Haoyu Wang,
• Abstract summary: We propose an automated fuzz testing method driven by a code knowledge graph and powered by an intelligent agent system.<n>The code knowledge graph is constructed through interprocedural program analysis, where each node in the graph represents a code entity.<n> CKGFuzzer achieved an average improvement of 8.73% in code coverage compared to state-of-the-art techniques.
• Score: 29.490817477791357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, the programming capabilities of large language models (LLMs) have garnered significant attention. Fuzz testing, a highly effective technique, plays a key role in enhancing software reliability and detecting vulnerabilities. However, traditional fuzz testing tools rely on manually crafted fuzz drivers, which can limit both testing efficiency and effectiveness. To address this challenge, we propose an automated fuzz testing method driven by a code knowledge graph and powered by an LLM-based intelligent agent system, referred to as CKGFuzzer. We approach fuzz driver creation as a code generation task, leveraging the knowledge graph of the code repository to automate the generation process within the fuzzing loop, while continuously refining both the fuzz driver and input seeds. The code knowledge graph is constructed through interprocedural program analysis, where each node in the graph represents a code entity, such as a function or a file. The knowledge graph-enhanced CKGFuzzer not only effectively resolves compilation errors in fuzz drivers and generates input seeds tailored to specific API usage scenarios, but also analyzes fuzz driver crash reports, assisting developers in improving code quality. By querying the knowledge graph of the code repository and learning from API usage scenarios, we can better identify testing targets and understand the specific purpose of each fuzz driver. We evaluated our approach using eight open-source software projects. The experimental results indicate that CKGFuzzer achieved an average improvement of 8.73% in code coverage compared to state-of-the-art techniques. Additionally, CKGFuzzer reduced the manual review workload in crash case analysis by 84.4% and successfully detected 11 real bugs (including nine previously unreported bugs) across the tested libraries.

Related papers

• Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models [49.214291813478695]
  Deep learning (DL) libraries, widely used in AI applications, often contain vulnerabilities like overflows and use buffer-free errors. Traditional fuzzing struggles with the complexity and API diversity of DL libraries. We propose DFUZZ, an LLM-driven fuzzing approach for DL libraries.
  arXiv  Detail & Related papers  (2025-01-08T07:07:22Z)
• FuzzDistill: Intelligent Fuzzing Target Selection using Compile-Time Analysis and Machine Learning [0.0]
  I present FuzzDistill, an approach that harnesses compile-time data and machine learning to refine fuzzing targets. I demonstrate the efficacy of my approach through experiments conducted on real-world software, demonstrating substantial reductions in testing time.
  arXiv  Detail & Related papers  (2024-12-11T04:55:58Z)
• A Combined Feature Embedding Tools for Multi-Class Software Defect and Identification [2.2020053359163305]
  We present CodeGraphNet, an experimental method that combines GraphCodeBERT and Graph Convolutional Network approaches. This method captures intricate relation- ships between features, providing for more exact identification and separation of vulnerabilities. The DeepTree model, which is a hybrid of a Decision Tree and a Neural Network, outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-11-26T17:33:02Z)
• FuzzCoder: Byte-level Fuzzing Test via Large Language Model [46.18191648883695]
  We propose to adopt fine-tuned large language models (FuzzCoder) to learn patterns in the input files from successful attacks. FuzzCoder can predict mutation locations and strategies locations in input files to trigger abnormal behaviors of the program.
  arXiv  Detail & Related papers  (2024-09-03T14:40:31Z)
• KGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution [59.20933707301566]
  Large Language Models (LLMs) are consistently improving at increasingly realistic software engineering (SE) tasks. In real-world software stacks, significant SE effort is spent developing foundational system software like the Linux kernel. To evaluate if ML models are useful while developing such large-scale systems-level software, we introduce kGym and kBench.
  arXiv  Detail & Related papers  (2024-07-02T21:44:22Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Chain of Targeted Verification Questions to Improve the Reliability of Code Generated by LLMs [10.510325069289324]
  We propose a self-refinement method aimed at improving the reliability of code generated by LLMs. Our approach is based on targeted Verification Questions (VQs) to identify potential bugs within the initial code. Our method attempts to repair these potential bugs by re-prompting the LLM with the targeted VQs and the initial code.
  arXiv  Detail & Related papers  (2024-05-22T19:02:50Z)
• Mining Action Rules for Defect Reduction Planning [14.40839500239476]
  We introduce CounterACT, a Counterfactual ACTion rule mining approach that can generate defect reduction plans without black-box models. We compare the effectiveness of CounterACT with the original action rule mining algorithm and six established defect reduction approaches on 9 software projects. Our results show that, compared to competing approaches, CounterACT's explainable plans achieve higher overlap scores at the release level.
  arXiv  Detail & Related papers  (2024-05-22T15:31:09Z)
• Prompt Fuzzing for Fuzz Driver Generation [6.238058387665971]
  We propose PromptFuzz, a coverage-guided fuzzer for prompt fuzzing. It iteratively generates fuzz drivers to explore undiscovered library code. PromptFuzz achieved 1.61 and 1.63 times higher branch coverage than OSS-Fuzz and Hopper, respectively.
  arXiv  Detail & Related papers  (2023-12-29T16:43:51Z)
• Zero-Shot Detection of Machine-Generated Codes [83.0342513054389]
  This work proposes a training-free approach for the detection of LLMs-generated codes. We find that existing training-based or zero-shot text detectors are ineffective in detecting code. Our method exhibits robustness against revision attacks and generalizes well to Java codes.
  arXiv  Detail & Related papers  (2023-10-08T10:08:21Z)
• HOPPER: Interpretative Fuzzing for Libraries [6.36596812288503]
  HOPPER can fuzz libraries without requiring any domain knowledge. It transforms the problem of library fuzzing into the problem of interpreter fuzzing.
  arXiv  Detail & Related papers  (2023-09-07T06:11:18Z)
• How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation [31.77886516971502]
  This study is the first in-depth study targeting the important issues of using LLMs to generate effective fuzz drivers. Our study evaluated 736,430 generated fuzz drivers, with 0.85 billion token costs ($8,000+ charged tokens) Our insights have been implemented to improve the OSS-Fuzz-Gen project, facilitating practical fuzz driver generation in industry.
  arXiv  Detail & Related papers  (2023-07-24T01:49:05Z)
• ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers [60.6418431624873]
  Large language models (LLMs) excel at implementing code from functionality descriptions but struggle with algorithmic problems. We propose ALGO, a framework that synthesizes Algorithmic programs with LLM-Generated Oracles to guide the generation and verify their correctness. Experiments show that when equipped with ALGO, we achieve an 8x better one-submission pass rate over the Codex model and a 2.6x better one-submission pass rate over CodeT.
  arXiv  Detail & Related papers  (2023-05-24T00:10:15Z)
• Fault-Aware Neural Code Rankers [64.41888054066861]
  We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it. Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
  arXiv  Detail & Related papers  (2022-06-04T22:01:05Z)
• GraphEye: A Novel Solution for Detecting Vulnerable Functions Based on Graph Attention Network [8.420666984519826]
  We propose a novel solution named GraphEye to identify whether a function of C/C++ code has vulnerabilities. VecCPG is a vectorization for the code property graph, which is proposed to characterize the key syntax and semantic features of the corresponding source code. GcGAT is a deep learning model based on the graph attention graph, which is proposed to solve the graph classification problem.
  arXiv  Detail & Related papers  (2022-02-05T07:03:15Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation [57.92972327649165]
  This work explores a deep learning approach to automatically learn the insecure patterns from code corpora. Because code naturally admits graph structures with parsing, we develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program.
  arXiv  Detail & Related papers  (2021-09-07T21:24:36Z)
• Break-It-Fix-It: Unsupervised Learning for Program Repair [90.55497679266442]
  We propose a new training approach, Break-It-Fix-It (BIFI), which has two key ideas. We use the critic to check a fixer's output on real bad inputs and add good (fixed) outputs to the training data. Based on these ideas, we iteratively update the breaker and the fixer while using them in conjunction to generate more paired data. BIFI outperforms existing methods, obtaining 90.5% repair accuracy on GitHub-Python and 71.7% on DeepFix.
  arXiv  Detail & Related papers  (2021-06-11T20:31:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.