ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning

• URL: http://arxiv.org/abs/2603.11226v1
• Date: Wed, 11 Mar 2026 18:49:45 GMT
• Title: ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning
• Abstract summary: We introduce ExecVerify, which goes beyond text imitation by incorporating verifiable white-box rewards derived from execution traces.<n>Experiments demonstrate that a 7B model trained with ExecVerify achieves performance comparable to 32B models on code reasoning benchmarks.
• Score: 9.137158235106941
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code LLMs still struggle with code execution reasoning, especially in smaller models. Existing methods rely on supervised fine-tuning (SFT) with teacher-generated explanations, primarily in two forms: (1) input-output (I/O) prediction chains and (2) natural-language descriptions of execution traces. However, intermediate execution steps cannot be explicitly verified during SFT, so the training objective can reduce to merely matching teacher explanations. Moreover, training data is typically collected without explicit control over task difficulty. We introduce ExecVerify, which goes beyond text imitation by incorporating verifiable white-box rewards derived from execution traces, including next-statement prediction and variable value/type prediction. Our work first builds a dataset with multiple difficulty levels via constraint-based program synthesis. Then, we apply reinforcement learning (RL) to reward correct answers about both intermediate execution steps and final outputs, aligning the training objective with semantic correctness at each execution step. Finally, we adopt a two-stage training pipeline that first enhances execution reasoning and then transfers to code generation. Experiments demonstrate that a 7B model trained with ExecVerify achieves performance comparable to 32B models on code reasoning benchmarks and improves pass@1 by up to 5.9\% on code generation tasks over strong post-training baselines.

Related papers

• TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning [5.703211775506213]
  We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains.<n>TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2.
  arXiv  Detail & Related papers  (2026-07-31T16:09:44Z)
• REVES: REvision and VErification--Augmented Training for Test-Time Scaling [53.197756110943395]
  We propose a two-stage iterative framework that alternates between online data/prompt augmentation and policy optimization.<n>We observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training.
  arXiv  Detail & Related papers  (2026-06-17T10:37:23Z)
• DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode [56.14374797825548]
  We introduce LLM-based pseudocode execution, which grounds prediction on more error-resilient pseudocode.<n>We propose DuET, a dual-execution framework that combines both approaches by functional majority voting.<n>On LiveCodeBench, DuET achieves the state-of-the-art performance, improving Pass@1 by 13.6 pp.
  arXiv  Detail & Related papers  (2026-04-13T14:18:58Z)
• Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards [76.49428173793386]
  LLMs must invoke multiple dependent APIs in the correct order while propagating intermediate outputs.<n>Training models to handle such faces two obstacles: existing environments focus on simple per-turn function calls with simulated data, and binary rewards.<n>We present a framework addressing both challenges. First, we construct a reinforcement learning environment backed by a large-scale cache of real API responses, enabling a data synthesis pipeline that samples valid multi-step orchestration traces.<n>Second, we propose a graduated reward design that decomposes correctness into atomic validity and orchestration.
  arXiv  Detail & Related papers  (2026-03-25T18:31:39Z)
• Generating Verifiable CoT from Execution-Traces [6.634229408414094]
  Chain-of-Thought prompting has shown promise, but current synthetic training data suffers from a critical weakness.<n>We address this by grounding CoT generation directly in program execution traces.<n>This execution-grounded approach ensures every reasoning step reflects what the program genuinely computes.
  arXiv  Detail & Related papers  (2025-11-28T07:43:43Z)
• Alignment with Fill-In-the-Middle for Enhancing Code Generation [56.791415642365415]
  We propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases.<n>Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench.
  arXiv  Detail & Related papers  (2025-08-27T03:15:53Z)
• CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL [28.43882967593511]
  Code large language models (LLMs) have become indispensable tools for building efficient and automated coding pipelines.<n>Existing models are typically post-trained using reinforcement learning (RL) from general-purpose LLMs using "human instruction-final answer" pairs.<n>We propose CodeBoost, a framework that enhances code LLMs purely from code snippets, without relying on human-annotated instructions.
  arXiv  Detail & Related papers  (2025-08-07T10:31:24Z)
• CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning [8.197518276987989]
  Code reasoning is a fundamental capability for large language models (LLMs) in the code domain.<n>Prior approaches mainly rely on supervised fine-tuning to improve performance in code reasoning tasks.<n>We argue this is due to two core issues: the low quality of training data and the limitations of supervised fine-tuning.<n>We propose CodeReasoner, a framework that spans both dataset construction and a two-stage training process.
  arXiv  Detail & Related papers  (2025-07-23T14:26:58Z)
• What I cannot execute, I do not understand: Training and Evaluating LLMs on Program Execution Traces [27.090845930270486]
  We study Execution Tuning (E.T.), a training procedure in which we explicitly model real-world program execution traces.<n>We train and evaluate models on different execution trace granularities (line and instruction-level) and strategies on the task of output prediction.
  arXiv  Detail & Related papers  (2025-02-10T14:42:13Z)
• Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension [18.919972400933393]
  We propose an advanced pretraining task, "Next Token Prediction+" Following this pretraining, both Code Llama and StarCoder, the prevalent code domain pretraining models, display significant improvements on our logically equivalent code selection task and the code completion task.
  arXiv  Detail & Related papers  (2024-04-13T03:11:07Z)
• Instruction Position Matters in Sequence Generation with Large Language Models [67.87516654892343]
  Large language models (LLMs) are capable of performing conditional sequence generation tasks, such as translation or summarization. We propose enhancing the instruction-following capability of LLMs by shifting the position of task instructions after the input sentences.
  arXiv  Detail & Related papers  (2023-08-23T12:36:57Z)
• LeTI: Learning to Generate from Textual Interactions [60.425769582343506]
  We explore LMs' potential to learn from textual interactions (LETI) that not only check their correctness with binary labels but also pinpoint and explain errors in their outputs through textual feedback. Our focus is the code generation task, where the model produces code based on natural language instructions. LETI iteratively fine-tunes the model, using the objective LM, on a concatenation of natural language instructions, LM-generated programs, and textual feedback.
  arXiv  Detail & Related papers  (2023-05-17T15:53:31Z)
• Soft-Labeled Contrastive Pre-training for Function-level Code Representation [127.71430696347174]
  We present textbfSCodeR, a textbfSoft-labeled contrastive pre-training framework with two positive sample construction methods. Considering the relevance between codes in a large-scale code corpus, the soft-labeled contrastive pre-training can obtain fine-grained soft-labels. SCodeR achieves new state-of-the-art performance on four code-related tasks over seven datasets.
  arXiv  Detail & Related papers  (2022-10-18T05:17:37Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.