Quality-Assured Fuzz Harness Generation via the Four Principles Framework

• URL: http://arxiv.org/abs/2605.21824v1
• Date: Wed, 20 May 2026 23:48:26 GMT
• Title: Quality-Assured Fuzz Harness Generation via the Four Principles Framework
• Abstract summary: We present QuartetFuzz, an autonomous harness-generation system that improves correctness throughout the generation process.<n>At its core is the Four Principles framework -- Logic Correctness (P1), API Protocol Compliance (P2), Security Boundary Respect (P3), and Entry Point Adequacy (P4)
• Score: 9.484544174862325
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fuzz testing is the dominant technique for finding memory-safety vulnerabilities in C/C++ software, yet its effectiveness hinges on the quality of fuzz harnesses -- the programs that bridge fuzzers and library APIs. A growing body of tools now automate harness generation, but none systematically ensures the correctness of produced harnesses: logic errors, API misuse, and lifecycle violations go undetected at the source level. As LLM-driven generation scales harness creation, uncontrolled quality turns scale into a liability. We present QuartetFuzz, an autonomous harness-generation system that systematically improves correctness throughout the generation process. At its core is the Four Principles framework -- Logic Correctness (P1), API Protocol Compliance (P2), Security Boundary Respect (P3), and Entry Point Adequacy (P4) -- the first source-level definition of harness correctness with mathematical specifications and implementable checks. We operationalize these principles in an autonomous LLM agent that produces harnesses satisfying P1-P4 through a generate-check-fix loop before any fuzzing begins. Deployed on 23 open-source projects spanning C/C++, Java, and JavaScript, the system submits 42 bug reports, of which 29 are fixed or confirmed upstream (including 3 CVEs) and only 2 are rejected (4.8% FP rate). During generation, the built-in P1/P2 checks automatically intercepted 58 harness-induced crashes that would otherwise have been false positives. Applied as a quality auditor to 586 existing production harnesses across 70 projects, the system identifies 53 violations (45 confirmed, 35 fixed). We release a dataset of 100 labeled harnesses for reproducible evaluation. Code and dataset are available at https://github.com/OwenSanzas/QuartetFuzz

Related papers

• FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs [46.58695049126903]
  We present FuzzingBrain-Bench, a benchmark for assessing AI models' ability to discover bugs in open-source software.<n>A model's performance on each challenge is scored based on the number of distinct crash signatures it produces.<n>Claude Opus 4.8 performs best, triggering crashes in 60 of 77 challenges and achieving a score of 196 out of 579.
  arXiv  Detail & Related papers  (2026-08-25T21:09:20Z)
• HarnessLLM: Rust Verification Harness Generation with Large Language Models [8.14069260667009]
  Rust's ownership model and type system offer strong memory safety guarantees, but unsafe code and runtime panics still present significant risks.<n>Formal verification is essential to ensure memory safety, but developing verification harnesses remains a challenging and manual task.<n>We present HarnessLLM, an automated workflow that leverages large language models to generate verification harnesses for Rust code directly from existing test suites.
  arXiv  Detail & Related papers  (2026-07-24T10:06:52Z)
• Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes [55.114939648705395]
  Large language models (LLMs) have made automated program repair increasingly practical for real-world bugs.<n>Bug reproduction tests (BRTs) help close this gap by turning a bug report into an executable, bug-specific signal.<n>We propose CoHarden, a co-generation framework that uses the Lax signal as an in-loop convergence criterion.
  arXiv  Detail & Related papers  (2026-07-22T07:30:07Z)
• TTHE: Test-Time Harness Evolution [50.26245555541721]
  Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time.<n>We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs.<n>During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces.<n>A judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs.
  arXiv  Detail & Related papers  (2026-07-09T05:53:39Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• FuzzAgent: Multi-Agent System for Evolutionary Library Fuzzing [8.054484953700365]
  Library fuzzing is essential for hardening the software supply chain, but adopting it at scale remains expensive.<n>We present FuzzAgent, a multi-agent system that turns library fuzzing into an evolutionary process.<n>We evaluate FuzzAgent on 20 real-world C/C++ libraries against four state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-05-14T06:21:33Z)
• Meta-Harness: End-to-End Optimization of Model Harnesses [52.31507076660471]
  We introduce Meta-Harness, an outer-loop system that searches over harness code for large language model (LLM) applications.<n>On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens.<n>On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models.
  arXiv  Detail & Related papers  (2026-03-30T05:33:50Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• Specification-Guided Repair of Arithmetic Errors in Dafny Programs using LLMs [79.74676890436174]
  We present an APR tool for Dafny that uses formal specifications as oracles for fault localization and repair.<n>We localize faults through a series of steps, which include using Hoare logic to determine the state of each statement within the program.<n>Our tool achieves 89.6% fault localization coverage and GPT-4o mini yields the highest repair success rate of 74.18%.
  arXiv  Detail & Related papers  (2025-07-04T15:36:12Z)
• May the Feedback Be with You! Unlocking the Power of Feedback-Driven Deep Learning Framework Fuzzing via LLMs [20.03968975178177]
  fuzz testing (Fuzzing) is a simple yet effective way to find bugs in Deep Learning (DL) frameworks.<n>We propose FUEL to effectively utilize the feedback information, which comprises two Large Language Models (LLMs): analysis LLM and generation LLM.<n>We show that FUEL can improve line code coverage of PyTorch and execution by 9.15% and 14.70% over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-06-21T08:51:53Z)
• CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph [29.490817477791357]
  We propose an automated fuzz testing method driven by a code knowledge graph and powered by an intelligent agent system.<n>The code knowledge graph is constructed through interprocedural program analysis, where each node in the graph represents a code entity.<n> CKGFuzzer achieved an average improvement of 8.73% in code coverage compared to state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-11-18T12:41:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.