StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning

• URL: http://arxiv.org/abs/2605.11922v1
• Date: Tue, 12 May 2026 10:36:56 GMT
• Title: StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
• Abstract summary: We propose StepCodeReasoner, a framework that introduces explicit intermediate execution-state supervision.<n>By automatically inserting structured print-based execution-trace anchors into code, the model is trained to predict runtime states at each step.<n>Our approach also improves code generation performance, demonstrating that explicit execution modeling enhances both code reasoning and code generation.
• Score: 16.3498616105159
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inconsistent reasoning. We propose StepCodeReasoner, a framework that introduces explicit intermediate execution-state supervision. By automatically inserting structured print-based execution-trace anchors into code, the model is trained to predict runtime states at each step, transforming code reasoning into a verifiable, stepwise execution modeling problem. Building on this execution-aware method, we introduce Bi-Level GRPO, a reinforcement learning algorithm for structured credit assignment at two levels: inter-trajectory, comparing alternative execution paths, and intra-trajectory, rewarding intermediate accuracy based on its impact on downstream correctness. Extensive experiments demonstrate that StepCodeReasoner achieves SOTA performance in code reasoning. In particular, our 7B model achieves 91.1\% on CRUXEval and 86.5\% on LiveCodeBench, outperforming the CodeReasoner-7B baseline (86.0\% and 77.7\%) and GPT-4o (85.6\% and 75.1\%). Furthermore, on the execution-trace benchmark REval, our model scores 82.9\%, outperforming baseline CodeReasoner-7B (72.3\%), its 14B counterpart (81.1\%), and GPT-4o (77.3\%). Additionally, our approach also improves code generation performance, demonstrating that explicit execution modeling enhances both code reasoning and code generation.

Related papers

• Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence [56.25095230687242]
  Coding agents often over-trust their own local validation routines and declare success on artifacts that satisfy surface checks.<n>This problem is especially acute in conversion, where prior evaluation is outcome-driven.<n>Across blind conversion attempts, the best system reaches 26.7--28.9% overall pass rate despite Spec pass rates up to 91.1%.<n>This suggests that failures stem more from contract-misaligned self-validation than from limited budget or backbone strength.
  arXiv  Detail & Related papers  (2026-05-27T19:57:15Z)
• Goedel-Code-Prover: Hierarchical Proof Search for Open State-of-the-Art Code Verification [34.98335927187393]
  Large language models (LLMs) can generate plausible code but offer limited guarantees of correctness.<n>We propose a hierarchical proof search framework for automated code verification in Lean4.<n>We train Goedel-Code-Prover-8B, a single unified policy for both decomposition and completion.
  arXiv  Detail & Related papers  (2026-03-18T18:42:04Z)
• How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs [49.61011897610774]
  How2Everything is a framework to evaluate and improve goal-conditioned procedure generation.<n>Our framework includes How2Mine, which mines 351K procedures from 980K web pages across 14 topics.<n>How2Score is an evaluation protocol that uses an LLM judge to detect whether a generation contains any critical failure that would prevent achieving the goal.
  arXiv  Detail & Related papers  (2026-02-09T15:47:14Z)
• Chain of Execution Supervision Promotes General Reasoning in Large Language Models [48.100128916029064]
  We introduce TracePile, a large-scale corpus of 2.6 million samples that transforms code execution into explicit, step-by-step chain-of-thought-style rationales.<n>We evaluate TracePile using three training setups: continue-pretraining, instruction tuning after pretraining, and two-stage finetuning.<n> Notably, TracePile boosts LLaMA3.1-8B by 7.1% on average across nine math datasets and delivers clear gains on LiveCodeBench, CRUX, and MMLU.
  arXiv  Detail & Related papers  (2025-10-24T02:21:11Z)
• CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment [98.87395842351627]
  Large Language Models (LLMs) excel at code generation by learning from vast code corpora.<n>A fundamental semantic gap remains between their training on textual patterns and the goal of functional correctness.<n>We propose CodeRL+, a novel approach that integrates execution semantics alignment into the RLVR training pipeline for code generation.
  arXiv  Detail & Related papers  (2025-10-21T09:48:06Z)
• CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs [14.437990879740191]
  We introduce CoreThink, a state-of-the-art Reasoning Layer built upon a novel reasoning method called General Symbolics.<n>This approach diverges from reasoning paradigms such as test-time scaling, Supervised Fine-Tuning (SFT), and Reinforcement Learning with Verifiable Rewards (RLVR)
  arXiv  Detail & Related papers  (2025-08-31T19:31:53Z)
• Posterior-GRPO: Rewarding Reasoning Processes in Code Generation [11.474187778340012]
  Reinforcement learning has significantly advanced code generation for large language models.<n>Current paradigms rely on outcome-based rewards from test cases, neglecting the quality of the intermediate reasoning process.<n>We introduce a unified framework that can effectively incorporate the quality of the reasoning process during RL.
  arXiv  Detail & Related papers  (2025-08-07T09:04:10Z)
• CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning [8.197518276987989]
  Code reasoning is a fundamental capability for large language models (LLMs) in the code domain.<n>Prior approaches mainly rely on supervised fine-tuning to improve performance in code reasoning tasks.<n>We argue this is due to two core issues: the low quality of training data and the limitations of supervised fine-tuning.<n>We propose CodeReasoner, a framework that spans both dataset construction and a two-stage training process.
  arXiv  Detail & Related papers  (2025-07-23T14:26:58Z)
• Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code [76.80306464249217]
  We propose TeaR, which aims at teaching LLMs to reason better.<n>TeaR leverages careful data curation and reinforcement learning to guide models in discovering optimal reasoning paths through code-related tasks.<n>We conduct extensive experiments using two base models and three long-CoT distillation models, with model sizes ranging from 1.5 billion to 32 billion parameters, and across 17 benchmarks spanning Math, Knowledge, Code, and Logical Reasoning.
  arXiv  Detail & Related papers  (2025-07-10T07:34:05Z)
• CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models [106.11371409170818]
  Large language models (LLMs) can act as agents with capabilities to self-refine and improve generated code autonomously. We propose CodeTree, a framework for LLM agents to efficiently explore the search space in different stages of the code generation process. Specifically, we adopted a unified tree structure to explicitly explore different coding strategies, generate corresponding coding solutions, and subsequently refine the solutions.
  arXiv  Detail & Related papers  (2024-11-07T00:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.