Human in the Loop for Fuzz Testing: Literature Review and the Road Ahead

• URL: http://arxiv.org/abs/2603.13411v1
• Date: Thu, 12 Mar 2026 12:39:16 GMT
• Title: Human in the Loop for Fuzz Testing: Literature Review and the Road Ahead
• Abstract summary: We propose a research agenda emphasizing future opportunities in (1) human monitoring, (2) human steering, and (3) human-LLM collaboration.<n>We call for a paradigm shift toward interactive, human-guided fuzzing systems that integrate expert insight with AI-powered automation.
• Score: 21.211596381653838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fuzz testing is one of the most effective techniques for detecting bugs and vulnerabilities in software. However, as the basis of fuzz testing, automated heuristics often fail to uncover deep or complex vulnerabilities. As a result, the performance of fuzz testing remains limited. One promising way to address this limitation is to integrate human expert guidance into the paradigm of fuzz testing. Even though some works have been proposed in this direction, there is still a lack of a systematic research roadmap for combining Human-in-the-Loop (HITL) and fuzz testing, hindering the potential for further enhancing fuzzing effectiveness. To bridge this gap, this paper outlines a forward-looking research roadmap for HITL for fuzz testing. Specifically, we highlight the promise of visualization techniques for interpretable fuzzing processes, as well as on-the-fly interventions that enable experts to guide fuzzing toward hard-to-reach program behaviors. Moreover, the rise of Large Language Models (LLMs) introduces new opportunities and challenges, raising questions about how humans can efficiently provide actionable knowledge, how expert meta-knowledge can be leveraged, and what roles humans should play in the intelligent fuzzing loop with LLMs. To address these questions, we survey existing work on HITL fuzz testing and propose a research agenda emphasizing future opportunities in (1) human monitoring, (2) human steering, and (3) human-LLM collaboration. We call for a paradigm shift toward interactive, human-guided fuzzing systems that integrate expert insight with AI-powered automation in the next-generation fuzzing ecosystem.

Related papers

• InsightQL: Advancing Human-Assisted Fuzzing with a Unified Code Database and Parameterized Query Interface [8.846926306547646]
  InsightQL is the first human-assisting framework for fuzz blocker analysis.<n>Powered by a unified database and an intuitive parameterized query interface, InsightQL aids developers in systematically extracting insights.<n>Our experiments on 14 popular real-world libraries from the FuzzBench benchmark demonstrate the effectiveness of InsightQL.
  arXiv  Detail & Related papers  (2025-10-06T14:18:35Z)
• SciML Agents: Write the Solver, Not the Solution [69.5021018644143]
  We introduce two new datasets: a diagnostic dataset of adversarial "misleading" problems; and a large-scale benchmark of 1,000 diverse ODE tasks.<n>We evaluate open- and closed-source LLM models along two axes: (i) unguided versus guided prompting with domain-specific knowledge; and (ii) off-the-shelf versus fine-tuned variants.<n>Preliminary results indicate that careful prompting and fine-tuning can yield a specialized LLM agent capable of reliably solving simple ODE problems.
  arXiv  Detail & Related papers  (2025-09-12T02:53:57Z)
• UQ: Assessing Language Models on Unsolved Questions [149.46593270027697]
  We introduce UQ, a testbed of 500 challenging, diverse questions sourced from Stack Exchange.<n>UQ is difficult and realistic by construction: unsolved questions are often hard and naturally arise when humans seek answers.<n>The top model passes UQ-validation on only 15% of questions, and preliminary human verification has already identified correct answers.
  arXiv  Detail & Related papers  (2025-08-25T01:07:59Z)
• LLAMA: Multi-Feedback Smart Contract Fuzzing Framework with LLM-Guided Seed Generation [56.84049855266145]
  We propose a Multi-feedback Smart Contract Fuzzing framework (LLAMA) that integrates evolutionary mutation strategies, and hybrid testing techniques.<n>LLAMA achieves 91% instruction coverage and 90% branch coverage, while detecting 132 out of 148 known vulnerabilities.<n>These results highlight LLAMA's effectiveness, adaptability, and practicality in real-world smart contract security testing scenarios.
  arXiv  Detail & Related papers  (2025-07-16T09:46:58Z)
• BandFuzz: An ML-powered Collaborative Fuzzing Framework [22.247404965632285]
  Collaborative fuzzing combines multiple individual fuzzers and dynamically chooses appropriate combinations for different programs.<n>We present BANDFUZZ, an ML-powered collaborative fuzzing framework that outperforms individual fuzzers without requiring additional computational resources.<n>We show that BANDFUZZ outperforms state-of-the-art collaborative fuzzing framework autofz and widely used individual fuzzers.
  arXiv  Detail & Related papers  (2025-07-14T22:37:21Z)
• HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance [71.5820853722963]
  Human-centered images often suffer from severe generic degradation during transmission and are prone to human motion blur (HMB)<n>We design a degradation pipeline that simulates the coexistence of HMB and generic noise, generating synthetic degraded data to train our proposed HAODiff.<n>For fair evaluation, we introduce MPII-Test, a benchmark rich in combined noise and HMB cases.
  arXiv  Detail & Related papers  (2025-05-26T09:24:11Z)
• Language Models Learn to Mislead Humans via RLHF [100.95201965748343]
  Language models (LMs) can produce errors that are hard to detect for humans, especially when the task is complex.<n>We study this phenomenon under a standard RLHF pipeline, calling it "U-SOPHISTRY" since it is Unintended by model developers.<n>Our results highlight an important failure mode of RLHF and call for more research in assisting humans to align them.
  arXiv  Detail & Related papers  (2024-09-19T14:50:34Z)
• FuzzCoder: Byte-level Fuzzing Test via Large Language Model [46.18191648883695]
  We propose to adopt fine-tuned large language models (FuzzCoder) to learn patterns in the input files from successful attacks. FuzzCoder can predict mutation locations and strategies locations in input files to trigger abnormal behaviors of the program.
  arXiv  Detail & Related papers  (2024-09-03T14:40:31Z)
• SoK: Prudent Evaluation Practices for Fuzzing [21.113311952857778]
  We systematically analyze the evaluation of 150 fuzzing papers published between 2018 and 2023. We study how existing guidelines are implemented and observe potential shortcomings and pitfalls. For example, when investigating reported bugs, we find a surprising disregard of the existing guidelines regarding statistical tests and systematic errors in fuzzing evaluations.
  arXiv  Detail & Related papers  (2024-05-16T16:10:41Z)
• Vulnerability Detection Through an Adversarial Fuzzing Algorithm [2.074079789045646]
  This project aims to increase the efficiency of existing fuzzers by allowing fuzzers to explore more paths and find more bugs in shorter amounts of time. adversarial methods are built on top of current evolutionary algorithms to generate test cases for further and more efficient fuzzing.
  arXiv  Detail & Related papers  (2023-07-21T21:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.