Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence

• URL: http://arxiv.org/abs/2608.31075v2
• Date: Tue, 01 Sep 2026 03:17:42 GMT
• Title: Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence
• Abstract summary: Reinforcement learning with verifiable rewards can substantially improve reasoning in mathematics and code.<n>But direct human supervision cannot keep pace with the scale and complexity of model-generated experience.<n>This paper studies how LRMs can continue to improve as human supervision gradually recedes from the learning loop.
• Score: 77.303673110294
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in large reasoning models (LRMs) have shown that reinforcement learning with verifiable rewards (RLVR) can substantially improve reasoning in mathematics and code, where outcomes can be checked automatically. Extending this progress to open-ended and agentic tasks remains difficult because reliable rewards are harder to obtain and direct human supervision cannot keep pace with the scale and complexity of model-generated experience. This paper studies how LRMs can continue to improve as human supervision gradually recedes from the learning loop. We examine two connected dimensions of this problem. The reward axis traces the development from per-instance human judgments to reusable verifiers and rewards that operate even without human feedback. The experience axis examines how learning can progress from human-curated tasks and environments toward self-generated curricula, constructed environments, and autonomous co-evolution. We connect these dimensions through a five-level ladder from L0 to L4 that identifies which parts of the learning process remain under continued human control. Our analysis further highlights the risks introduced by increasingly autonomous rewards and experience generation, including reward hacking, feedback drift, curriculum collapse, and environment errors. Consequently, we also provide the evaluation around three complementary objects: policy capability, feedback fidelity, and experience quality. This analysis provides a structured account of current approaches to scaling LRMs beyond human supervision and the open problems involved in developing self-sustaining learning systems toward superintelligence. Furthermore, we maintain a continuously updated \href{https://github.com/visitworld123/Awesome-Scaling-LRM-Beyond-Human-Supervision}{GitHub repository} to track the latest advances.

Related papers

• Aspire: Can Models Self-Evolve from Vague Goals? [61.65007006747275]
  We introduce ASPIRE, a benchmark for vague-goal-driven self-evolution.<n>ASPIRE supports both model-weight and agent-harness evolution in a unified interactive environment.<n>Our experiments show that vague goals redirect search effort toward goal interpretation.
  arXiv  Detail & Related papers  (2026-08-31T17:14:59Z)
• Training Skills Like Parameters via Self-Supervised Semantic Diffusion [15.133304131514713]
  Large Language Models (LLMs) often fall short of human experts in highly specialized, open-ended domains such as creative screenwriting.<n>We propose a novel, unsupervised self-evolving agent framework inspired by the corruption-and-reconstruction paradigm of diffusion models.<n>We evaluate our framework on the challenging task of short drama screenwriting.
  arXiv  Detail & Related papers  (2026-07-30T01:03:59Z)
• TangramSR: Can Vision-Language Models Reason in Continuous Geometric Space? [11.222572150508332]
  Humans excel at spatial reasoning tasks like Tangram puzzle assembly through cognitive processes involving mental rotation, iterative refinement, and visual feedback.<n>However, comprehensive experiments across five representative Vision-Language Models (VLMs) reveal systematic failures in continuous geometric reasoning.<n>We introduce a test-time self-refinement framework that combines in-context learning (ICL) with reward-guided feedback loops, inspired by human cognitive processes.
  arXiv  Detail & Related papers  (2026-02-05T11:49:30Z)
• Guided Self-Evolving LLMs with Minimal Human Supervision [53.111086364268566]
  Unguided self-evolving systems often plateau quickly or degrade as training.<n>R-Few is a guided Self-Play Challenger acquirer framework that incorporates lightweight human oversight through in-context grounding and mixed training.<n>R-Few achieves consistent and iterative improvements across math and general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T07:06:11Z)
• Absolute Zero: Reinforced Self-play Reasoning with Zero Data [57.30662797376754]
  Reinforcement learning with verifiable rewards (RLVR) has shown promise in enhancing the reasoning capabilities of large language models.<n>We introduce the Absolute Zero Reasoner (AZR), a system that self-evolves its training curriculum and reasoning ability.<n>AZR achieves overall SOTA performance on coding and mathematical reasoning tasks, outperforming existing zero-setting models.
  arXiv  Detail & Related papers  (2025-05-06T09:08:00Z)
• Emergent Abilities in Large Language Models: A Survey [9.50669909278749]
  Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence.<n>The scaling of these models has been linked to various so-called emergent abilities that were previously unobserved.<n>These abilities range from advanced reasoning and in-context learning to coding and problem-solving.<n>Despite their transformative potential, emergent abilities remain poorly understood, leading to misconceptions about their definition, nature, predictability, and implications.
  arXiv  Detail & Related papers  (2025-02-28T01:20:01Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.