Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus

• URL: http://arxiv.org/abs/2603.29292v1
• Date: Tue, 31 Mar 2026 05:55:17 GMT
• Title: Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus
• Abstract summary: ConSelf is a self-improving approach built upon two key ideas.<n>First, we introduce code semantic entropy, a novel metric that measures problem-level uncertainty.<n>Second, we present consensus-driven direct preference optimization (Con-DPO), a preference-based fine-tuning method.
• Score: 24.61607166744383
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Improving the code generation capabilities of large language models (LLMs) typically relies on supervised fine-tuning or preference optimization, both of which require costly external resources such as powerful teacher models or reliable test units. However, in real-world scenarios, it is much harder to obtain reference solutions and test oracles than problem descriptions and test inputs. In this paper, we tackle a challenging yet realistic question: Can a code language model improve itself without access to a superior teacher and a test oracle? To answer this, we propose ConSelf, a self-improving approach built upon two key ideas. First, we introduce code semantic entropy, a novel metric that measures problem-level uncertainty by assessing the functional diversity of program behaviors, enabling a curriculum construction with the most learnable problems. Second, we present consensus-driven direct preference optimization (Con-DPO), a preference-based fine-tuning method that weights each preference pair by its behavioral consensus, thereby mitigating the impact of noisy self-generated supervision. Experiments on various benchmarks and backbone LLMs demonstrate that ConSelf significantly outperforms baselines, validating the effectiveness of semantic entropy-based curriculum construction and consensus-driven optimization in improving code generation without external supervision.

Related papers

• INSPIRE: An Internalize-Then-Improve Approach for Example-Driven Mathematical Reasoning [61.60016491769943]
  In human mathematics education, example-based reasoning such as constructing counterexamples to test theorem boundaries reflects deep conceptual understanding.<n>We propose INSPIRE, an Internalize-Then-Improve approach combining Reference-Guided Student Internalization (RGSI) with a stage-wise preference training strategy.<n>Experiments across multiple model scales and families demonstrate consistent improvements, even surpassing larger open-source models.
  arXiv  Detail & Related papers  (2026-08-27T03:24:03Z)
• ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization [28.464748466548965]
  ACE is a self-evolving code generation framework based on a solver--adversary architecture.<n>It prioritizes active failure discovery through execution-centric supervision.<n>ACE consistently outperforms strong solver--verifier baselines.
  arXiv  Detail & Related papers  (2026-04-17T07:20:01Z)
• Enhancing LLM Problem Solving via Tutor-Student Multi-Agent Interaction [0.0]
  We propose an autonomous coding problem domain where two agents instantiated from the same Large Language Model (LLM) assigned asymmetric roles.<n>In our proposed framework (PETITE), we aim to extract better problem-solving performance from one model by structuring its interaction through complementary roles.<n>Results show that our model achieves similar or higher accuracy while consuming significantly fewer tokens.
  arXiv  Detail & Related papers  (2026-04-10T03:56:49Z)
• TAROT: Test-driven and Capability-adaptive Curriculum Reinforcement Fine-tuning for Code Generation with Large Language Models [26.385183692191873]
  Large Language Models (LLMs) are changing the coding paradigm, yet synthesizingically sophisticated and robust code still remains a critical challenge.<n>We propose Test-driven and cApability-adaptive cuRriculum reinfOrcement fineTuning (TAROT) to address this need.
  arXiv  Detail & Related papers  (2026-02-17T09:29:18Z)
• Reasoning Distillation and Structural Alignment for Improved Code Generation [0.6933020649563103]
  This work distills the reasoning capabilities of a large language model into a smaller, more efficient model that is faster and cheaper to deploy.<n>Our approach trains the model to emulate the reasoning and problem-solving abilities of the VLLM by learning to identify correct solution pathways.<n> Experimental results show that our fine-tuned model, developed through a cheap and simple to implement process, significantly outperforms our baseline model in terms of pass@1, average data flow, and average syntax match metrics.
  arXiv  Detail & Related papers  (2025-10-20T14:47:47Z)
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning [65.20602712957725]
  Caco is a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-CoT reasoning data.<n>Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.
  arXiv  Detail & Related papers  (2025-10-05T07:59:24Z)
• Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs [102.48588475875749]
  We introduce Generative Self-Refinement (GSR), a novel parallel test-time scaling framework.<n>GSR generates a set of candidate responses in parallel and then performs self-refinement to synthesize a new superior solution.<n>We show that our method achieves state-of-the-art performance across five mathematical benchmarks.
  arXiv  Detail & Related papers  (2025-08-27T06:51:48Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks.<n>However, improvement is plateauing due to the exhaustion of readily available high-quality data.<n>We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• CodeDPO: Aligning Code Models with Self Generated and Verified Source Code [52.70310361822519]
  We propose CodeDPO, a framework that integrates preference learning into code generation to improve two key code preference factors: code correctness and efficiency.<n>CodeDPO employs a novel dataset construction method, utilizing a self-generation-and-validation mechanism that simultaneously generates and evaluates code and test cases.
  arXiv  Detail & Related papers  (2024-10-08T01:36:15Z)
• SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models [54.78329741186446]
  We propose a novel paradigm that uses a code-based critic model to guide steps including question-code data construction, quality control, and complementary evaluation. Experiments across both in-domain and out-of-domain benchmarks in English and Chinese demonstrate the effectiveness of the proposed paradigm.
  arXiv  Detail & Related papers  (2024-08-28T06:33:03Z)
• Wait, that's not an option: LLMs Robustness with Incorrect Multiple-Choice Options [2.1184929769291294]
  This work introduces a novel framework for evaluating LLMs' capacity to balance instruction-following with critical reasoning.<n>We show that post-training aligned models often default to selecting invalid options, while base models exhibit improved refusal capabilities that scale with model size.<n>We additionally conduct a parallel human study showing similar instruction-following biases, with implications for how these biases may propagate through human feedback datasets used in alignment.
  arXiv  Detail & Related papers  (2024-08-27T19:27:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.