PPO guided Agentic Pipeline for Adaptive Prompt Selection and Test Case Generation

• URL: http://arxiv.org/abs/2605.00942v1
• Date: Fri, 01 May 2026 06:19:51 GMT
• Title: PPO guided Agentic Pipeline for Adaptive Prompt Selection and Test Case Generation
• Abstract summary: We present a novel approach for generating test cases using a reinforcement learning-driven agentic framework.<n>In Phase I, a ToT-guided optimization agent partitions and minimizes the source code by removing redundancies.<n>In Phase II, a PPO-based policy network is trained to solve the problem of selecting prompts among eight different prompting techniques.<n>The PPO agent receives rewards based on a combination of increases in line and branch coverages, penalties for unexplored branches, and rewards for reducing source code length.
• Score: 0.30786914102688595
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Developing effective test cases capable of thoroughly exercising large-scale software systems is inherently difficult, especially if such systems have voluminous, complex, and deeply nested source codes. In this work, we present a novel approach for generating test cases using a reinforcement learning-driven agentic framework where Proximal Policy Optimization (PPO) is coupled with an LLM engine to guide prompt selection during test generation. Our approach consists of two phases. In Phase I, the ToT-guided optimization agent partitions and minimizes the source code by removing redundancies without changing the functional behavior of the source code. In Phase II, a PPO-based policy network is trained to solve the problem of selecting prompts among eight different prompting techniques, such as Boundary Value Analysis, Random Fuzzing, etc., based on the inputted 11-dimensional state vector representing the source code complexity metrics and live coverage metrics to direct the LLM engine towards exploring unvisited paths in the program. The PPO agent receives rewards based on a combination of increases in line and branch coverages, penalties for unexplored branches, and rewards for reducing source code length. From experiments conducted on twenty benchmark programs, it is evident that the proposed approach, PPO-LLM, outperforms CBMC, kS-LLM, and kS-LLM++ in terms of branch and line coverage in almost all cases, for various loop bound values ranging from BOUND~1 to BOUND~2000. While at BOUND~1, the coverage of branches is 100\% using PPO-LLM on the PALS suite, in comparison, it is around 86.8\% using kS-LLM++. This confirms that adaptive prompt selection driven by PPO substantially outperforms static prompting strategies on PALS type programs.

Related papers

• The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
  We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
  arXiv  Detail & Related papers  (2026-08-07T02:15:39Z)
• APCD: Adaptive Path-Contrastive Decoding for Reliable Large Language Model Generation [9.34156158035883]
  Large language models (LLMs) often suffer from hallucinations due to error accumulation in autoregressive decoding.<n>We propose Adaptive Path-Contrastive Decoding (APCD), a multi-path decoding framework that improves output reliability through adaptive exploration and controlled path interaction.
  arXiv  Detail & Related papers  (2026-05-10T11:57:39Z)
• PARM: Pipeline-Adapted Reward Model [60.769414637325326]
  Reward models (RMs) are central to aligning large language models (LLMs) with human preferences, powering advanced decoding strategies.<n>While most prior work focuses on single-step generation, real-world applications increasingly adopt multi-stage pipelines.<n>We investigate this through code generation for optimization, constructing a pipeline that integrates reward models into both formulation and solution stages.
  arXiv  Detail & Related papers  (2026-04-20T14:29:08Z)
• SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks [41.49967840381499]
  Sequence-Level PPO (SPPO) is a scalable algorithm that harmonizes the sampletextual efficiency of PPO with the stability of outcome-based updates.<n>SPPO significantly surpasses standard PPO and matches the performance of computation-heavy group-based methods.
  arXiv  Detail & Related papers  (2026-04-10T01:58:21Z)
• POLCA: Stochastic Generative Optimization with LLM [26.342554900977003]
  We introduce Prioritized Optimization with Local Contextualion (POLCA)<n>POLCA is a scalable framework designed to handle parameterity in optimization.<n>We demonstrate POLCA achieves robust, sample and time-efficient performance.
  arXiv  Detail & Related papers  (2026-03-16T03:07:44Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference [4.955827313214081]
  Prefill-Decode (P/D) disaggregation has emerged as a widely adopted optimization strategy for Large Language Model (LLM) inference.<n>There currently exists no well-established methodology for determining the optimal number of P/D hardware resources.<n>We propose a hybrid approach that combines theoretical modeling with empirical benchmarking.
  arXiv  Detail & Related papers  (2026-03-05T01:41:09Z)
• GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer [51.31263673158136]
  GFlowPO casts prompt search as a posterior inference problem over latent prompts regularized by a meta-prompted reference-LM prior.<n>GFlowPO consistently outperforms recent discrete prompt optimization baselines.
  arXiv  Detail & Related papers  (2026-02-03T10:30:03Z)
• UPA: Unsupervised Prompt Agent via Tree-Based Search and Selection [29.332488308247623]
  Unsupervised Prompt Agent (UPA) realizes structured search and selection without relying on supervised feedback.<n>UPA first performs path-wise Bayesian aggregation of local comparisons to filter candidates under uncertainty, followed by global tournament-style comparisons to infer latent prompt quality.
  arXiv  Detail & Related papers  (2026-01-30T18:39:09Z)
• Solving the Granularity Mismatch: Hierarchical Preference Learning for Long-Horizon LLM Agents [56.625878022978945]
  Large Language Models (LLMs) as autonomous agents are increasingly tasked with solving complex, long-horizon problems.<n>Direct Preference Optimization (DPO) provides a signal that is too coarse for precise credit assignment, while step-level DPO is often too myopic to capture the value of multi-step behaviors.<n>We introduce Hierarchical Preference Learning (HPL), a hierarchical framework that optimize LLM agents by leveraging preference signals at multiple, synergistic granularities.
  arXiv  Detail & Related papers  (2025-09-26T08:43:39Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Surpassing legacy approaches to PWR core reload optimization with single-objective Reinforcement learning [0.0]
  We have developed methods based on Deep Reinforcement Learning (DRL) for both single- and multi-objective optimization. In this paper, we demonstrate the advantage of our RL-based approach, specifically using Proximal Policy Optimization (PPO) PPO adapts its search capability via a policy with learnable weights, allowing it to function as both a global and local search method.
  arXiv  Detail & Related papers  (2024-02-16T19:35:58Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.