Related papers: GASP: Guided Asymmetric Self-Play For Coding LLMs

GASP: Guided Asymmetric Self-Play For Coding LLMs

URL: http://arxiv.org/abs/2603.15957v1
Date: Mon, 16 Mar 2026 22:13:19 GMT
Title: GASP: Guided Asymmetric Self-Play For Coding LLMs
Abstract summary: Asymmetric self-play has emerged as a promising paradigm for post-training large language models.<n>We propose Guided Asymmetric Self-Play (GASP), where grounding is provided by real-data goalpost questions.<n>We improve pass@20 on LiveCodeBench (LCB) by 2.5% over unguided asymmetric self-play.
Score: 37.79170066221302
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Asymmetric self-play has emerged as a promising paradigm for post-training large language models, where a teacher continually generates questions for a student to solve at the edge of the student's learnability. Although these methods promise open-ended data generation bootstrapped from no human data, they suffer from one major problem: not all problems that are hard to solve are interesting or informative to improve the overall capabilities of the model. Current asymmetric self-play methods are goal-agnostic with no real grounding. We propose Guided Asymmetric Self-Play (GASP), where grounding is provided by real-data goalpost questions that are identified to pose a hard exploration challenge to the model. During self-play, the teacher first generates an easier variant of a hard question, and then a harder variant of that easier question, with the goal of gradually closing the gap to the goalpost throughout training. Doing so, we improve pass@20 on LiveCodeBench (LCB) by 2.5% over unguided asymmetric self-play, and through the curriculum constructed by the teacher, we manage to solve hard goalpost questions that remain out of reach for all baselines.

Related papers

Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematics [4.56953073429622]
Training data is scarce, ground-truth reasoning traces are usually unavailable, and models often exhibit an apparent ceiling beyond which additional data yields no further improvement.<n>We study these difficulties in a controlled setting, fine-tuning Qwen2.5-Math-7B on competition mathematics (AIME)<n>We propose a self-evolving curriculum that, each round, evaluates the current checkpoint, seeds QbQ from the problems it can mostly get right, and trains on the resulting variants.
arXiv Detail & Related papers (2026-08-02T22:19:01Z)
Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL [76.45061154544568]
Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels.<n>Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood.<n>We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted.
arXiv Detail & Related papers (2026-05-21T09:19:23Z)
Scaling Self-Play with Self-Guidance [62.13619253976748]
Self-Guided Self-Play (SGS) is a self-play algorithm that guides the Conjecturer away from degeneracy.<n>SGS surpasses the solve rate of our strongest baseline in fewer than 80 rounds of self-play.
arXiv Detail & Related papers (2026-04-22T05:50:23Z)
Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability [25.507069397981194]
We show that it is possible to realize bi-level meta-RL that unlocks learning under sparse, binary rewards by sharpening a latent capacity of pretrained models to generate useful stepping stones.<n>Our results suggest that the ability to generate useful stepping stones does not require the ability to actually solve the hard problems.
arXiv Detail & Related papers (2026-01-26T18:46:56Z)
DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution [38.660154251426505]
We introduce DARC (Decoupled Asymmetric Reasoning Curriculum), a two-stage framework that stabilizes the self-evolution process.<n>We show that DARC is model-agnostic, yielding an average improvement of 10.9 points across nine reasoning benchmarks and three backbone models.
arXiv Detail & Related papers (2026-01-20T09:12:27Z)
Self-Questioning Language Models [58.73276539661649]
We propose an asymmetric self-play framework where a proposer is given the topic and generates a question for a solver.<n>Both the proposer and solver are trained via reinforcement learning.<n>We study this asymmetric self-play framework on three benchmarks: three-digit multiplication, algebra problems from the OMEGA benchmark, and programming problems from Codeforces.
arXiv Detail & Related papers (2025-08-05T17:51:33Z)
TOFU: A Task of Fictitious Unlearning for LLMs [99.92305790945507]
Large language models trained on massive corpora of data from the web can reproduce sensitive or private data raising both legal and ethical concerns. Unlearning, or tuning models to forget information present in their training data, provides us with a way to protect private data after training. We present TOFU, a benchmark aimed at helping deepen our understanding of unlearning.
arXiv Detail & Related papers (2024-01-11T18:57:12Z)
R-Tuning: Instructing Large Language Models to Say `I Don't Know' [66.11375475253007]
Large language models (LLMs) have revolutionized numerous domains with their impressive performance but still face their challenges. Previous instruction tuning methods force the model to complete a sentence no matter whether the model knows the knowledge or not. We present a new approach called Refusal-Aware Instruction Tuning (R-Tuning) Experimental results demonstrate R-Tuning effectively improves a model's ability to answer known questions and refrain from answering unknown questions.
arXiv Detail & Related papers (2023-11-16T08:45:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.