R1-Fuzz: Specializing Language Models for Textual Fuzzing via Reinforcement Learning

• URL: http://arxiv.org/abs/2509.20384v1
• Date: Sun, 21 Sep 2025 15:21:43 GMT
• Title: R1-Fuzz: Specializing Language Models for Textual Fuzzing via Reinforcement Learning
• Authors: Jiayi Lin, Liangcai Su, Junzhe Li, Chenxiong Qian,
• Abstract summary: Fuzzing is effective for vulnerability discovery but struggles with complex targets such as compilers, interpreters, and database engines.<n>We propose R1-Fuzz, a framework that leverages reinforcement learning (RL) to specialize cost-efficient language models and integrate them for fuzzing input generation.<n>R1-Fuzz achieves up to 75% higher coverage than state-of-the-art fuzzers and discovers 29 previously unknown vulnerabilities.
• Score: 7.526332397353976
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Fuzzing is effective for vulnerability discovery but struggles with complex targets such as compilers, interpreters, and database engines, which accept textual input that must satisfy intricate syntactic and semantic constraints. Although language models (LMs) have attracted interest for this task due to their vast latent knowledge and reasoning potential, their practical adoption has been limited. The major challenges stem from insufficient exploration of deep program logic among real-world codebases, and the high cost of leveraging larger models. To overcome these challenges, we propose R1-Fuzz, the first framework that leverages reinforcement learning (RL) to specialize cost-efficient LMs and integrate them for complex textual fuzzing input generation. R1-Fuzz introduces two key designs: coverage-slicing-based question construction and a distance-based reward calculation. Through RL-based post-training of a model with our constructed dataset, R1-Fuzz designs a fuzzing workflow that tightly integrates LMs to reason deep program semantics during fuzzing. Evaluations on diverse real-world targets show that our design enables a small model, named R1-Fuzz-7B, to rival or even outperform much larger models in real-world fuzzing. Notably, R1-Fuzz achieves up to 75\% higher coverage than state-of-the-art fuzzers and discovers 29 previously unknown vulnerabilities, demonstrating its practicality.

Related papers

• MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval [50.30107119622642]
  Large Language Models (LLMs) excel at reasoning and generation but are inherently limited by static pretraining data.<n>Retrieval-Augmented Generation (RAG) addresses this issue by grounding LLMs in external knowledge.<n>MarAG-R1 is a reinforcement-learned multi-tool RAG framework that enables LLMs to dynamically coordinate multiple retrieval mechanisms.
  arXiv  Detail & Related papers  (2025-10-31T15:51:39Z)
• AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models [12.484537674896908]
  We propose AR$2$ (Adversarial Reinforcement Learning for Abstract Reasoning), a novel framework explicitly designed to enhance the abstraction abilities of large language models (LLMs)<n>AR$2$ employs a teacher model to transform kernel problems into narrative-rich, challenging descriptions without changing their fundamental logic.<n>A student coding model is trained to solve these complex narrative problems by extracting their underlying computational kernels.
  arXiv  Detail & Related papers  (2025-08-27T17:26:44Z)
• Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning [3.364797975300393]
  We present Observe-R1, a novel framework aimed at enhancing the reasoning capabilities of multimodal large language models (MLLMs)<n>We construct the NeuraLadder dataset, which is organized and sampled according to the difficulty and complexity of data samples for RL training.<n>Experiments with the Qwen2.5-VL-3B and Qwen2.5-VL-7B models on 20k samples from the NeuraLadder dataset show that Observe-R1 outperforms a series of larger reasoning models on both reasoning and general benchmarks.
  arXiv  Detail & Related papers  (2025-05-18T14:08:03Z)
• R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model [70.77691645678804]
  We present the first successful replication of emergent characteristics for multimodal reasoning on only a non-SFT 2B model.<n>Our model achieves 59.47% accuracy on CVBench, outperforming the base model by approximately 30% and exceeding both SFT setting by 2%.<n>In addition, we share our failed attempts and insights in attempting to achieve R1-like reasoning using RL with instruct models.
  arXiv  Detail & Related papers  (2025-03-07T04:21:47Z)
• New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration [49.180693704510006]
  Referring Expression (REC) is a cross-modal task that evaluates the interplay of language understanding, image comprehension, and language-to-image grounding.<n>It serves as an essential testing ground for Multimodal Large Language Models (MLLMs)
  arXiv  Detail & Related papers  (2025-02-27T13:58:44Z)
• SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models [74.40683913645731]
  Zero-shot multi-label recognition (MLR) with Vision-Language Models (VLMs) faces significant challenges without training data, model tuning, or architectural modifications.<n>Our work proposes a novel solution treating VLMs as black boxes, leveraging scores without training data or ground truth.<n>Analysis of these prompt scores reveals VLM biases and AND''/OR' signal ambiguities, notably that maximum scores are surprisingly suboptimal compared to second-highest scores.
  arXiv  Detail & Related papers  (2025-02-24T07:15:05Z)
• RIRO: Reshaping Inputs, Refining Outputs Unlocking the Potential of Large Language Models in Data-Scarce Contexts [0.0]
  Large language models (LLMs) have significantly advanced natural language processing, excelling in areas like text generation, summarization, and question-answering.<n>Despite their capabilities, these models face challenges when fine-tuned on small, domain-specific datasets.<n>We introduce RIRO, a novel two-layer architecture designed to improve performance in data-scarce environments.
  arXiv  Detail & Related papers  (2024-12-15T15:48:37Z)
• U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF [10.81723269312202]
  Mixture-of-Experts (MoE) have been proposed as an energy efficient path to larger and more capable language models. We benchmark our proposed model on a large scale inner-source dataset (160k hours)
  arXiv  Detail & Related papers  (2024-04-25T08:34:21Z)
• MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning [63.80739044622555]
  We introduce MuSR, a dataset for evaluating language models on soft reasoning tasks specified in a natural language narrative. This dataset has two crucial features. First, it is created through a novel neurosymbolic synthetic-to-natural generation algorithm. Second, our dataset instances are free text narratives corresponding to real-world domains of reasoning.
  arXiv  Detail & Related papers  (2023-10-24T17:59:20Z)
• Unsupervised Object-Centric Learning with Bi-Level Optimized Query Slot Attention [26.25900877220557]
  Slot-Attention module has played an important role with its simple yet effective design and fostered many powerful variants. We propose to address these issues by initializing Slot-Attention modules with learnable queries and (2) optimizing the model with bi-level optimization. Our model achieves state-of-the-art results on both synthetic and complex real-world datasets in unsupervised image segmentation and reconstruction.
  arXiv  Detail & Related papers  (2022-10-17T12:14:59Z)
• ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering [70.6359636116848]
  We propose a new large-scale dataset, ConvFinQA, to study the chain of numerical reasoning in conversational question answering. Our dataset poses great challenge in modeling long-range, complex numerical reasoning paths in real-world conversations.
  arXiv  Detail & Related papers  (2022-10-07T23:48:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.