ASI-Bench: At the Dawn of Artificial Superintelligence

• URL: http://arxiv.org/abs/2608.17271v1
• Date: Tue, 18 Aug 2026 01:59:00 GMT
• Title: ASI-Bench: At the Dawn of Artificial Superintelligence
• Abstract summary: ASI-Bench is the first benchmark to jointly evaluate AI systems' capabilities of innovative exploration and autonomous scientific execution.<n>It contains 60 project-level research tasks across 11 scientific domains and progressively reduces methodological guidance.<n>Across 18 state-of-the-art agent--model configurations, the average score drops from 50.91 with full methodological guidance to 29.10 with only the method specified.
• Score: 46.937202634067766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial superintelligence (ASI) requires AI to move beyond mastering existing knowledge toward exploring the unknown, creating new knowledge, and turning new ideas into verifiable results. However, the capabilities of today's AI systems are still largely built on learning, compressing, and applying existing human knowledge. Accordingly, existing benchmarks primarily test whether AI can produce correct answers based on learned knowledge, or whether it can complete tasks under extensive human guidance. We therefore introduce ASI-Bench, the first benchmark to jointly evaluate AI systems' capabilities of innovative exploration and autonomous scientific execution across general research domains, and the first to progressively withdraw human methodological guidance within the same research project to test how far AI can proceed on its own. Built by over 40 experts with the cost of 31,000+ human hours, ASI-Bench contains 60 project-level research tasks across 11 scientific domains and progressively reduces methodological guidance to test whether AI can independently select methods, conduct research, and produce verifiable results. All tasks undergo expert review, AI-assisted auditing, sandbox execution, and scorer validation. Across 18 state-of-the-art agent--model configurations, the average score drops from 50.91 with full methodological guidance to 29.10 with only the method specified and 26.62 when agents must determine the method themselves. This sharp decline shows that current systems remain heavily dependent on human guidance and are still far from autonomously conducting end-to-end, project-level scientific research. ASI-Bench is open to the world. We invite researchers and builders everywhere to contribute new tasks, challenge the limits of today's AI, and help accelerate humanity's collective path toward artificial superintelligence at https://asibench.apexin.ai/submit.

Related papers

• Atria Dawn: The Dawn of Agentic Superintelligence [237.73474618289063]
  We introduce Atria Dawn Preview, a foundation agentic language model designed for scientific research and engineering.<n>We examine the real research-and-development process behind this model as a case study of human--AI collaboration.
  arXiv  Detail & Related papers  (2026-09-14T16:22:30Z)
• When AI Meets Science: Research Diversity, Interdisciplinarity, Visibility, and Retractions across Disciplines in a Global Surge [0.429202654064509]
  We analyze over 227 million scholarly works from the OpenAlex collection.<n>We document differences in the timing and extent of AI adoption across domains.<n>We discuss how best research practices could boost the benefits of AI adoption.
  arXiv  Detail & Related papers  (2026-05-07T11:23:23Z)
• ASI-Evolve: AI Accelerates AI [42.885465604426884]
  ASI-Evolve is a framework for AI-for-AI research that closes the loop through a learn-design-experiment-analyze cycle.<n>To our knowledge, ASI-Evolve is the first unified framework to demonstrate AI-driven discovery across three central components of AI development.
  arXiv  Detail & Related papers  (2026-03-31T12:04:17Z)
• Deep Research of Deep Research: From Transformer to Agent, From AI to AI for Science [2.316539232916599]
  We articulate a clear and precise definition of deep research and unify perspectives from industry's deep research and academia's AI for Science.<n>We position LLMs and Stable Diffusion as the twin pillars of generative AI, and lay out a roadmap evolving from the Transformer to agents.
  arXiv  Detail & Related papers  (2026-03-30T12:29:47Z)
• The next question after Turing's question: Introducing the Grow-AI test [51.56484100374058]
  This study aims to extend the framework for assessing artificial intelligence, called GROW-AI.<n>GROW-AI is designed to answer the question "Can machines grow up?" -- a natural successor to the Turing Test.<n>The originality of the work lies in the conceptual transposition of the process of "growing" from the human world to that of artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-22T10:19:42Z)
• SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam? [51.112225746095746]
  We introduce X-Master, a tool-augmented reasoning agent designed to emulate human researchers.<n>X-Masters sets a new state-of-the-art record on Humanity's Last Exam with a score of 32.1%.
  arXiv  Detail & Related papers  (2025-07-07T17:50:52Z)
• AI4Research: A Survey of Artificial Intelligence for Scientific Research [55.5452803680643]
  We present a comprehensive survey on AI for Research (AI4Research)<n>We first introduce a systematic taxonomy to classify five mainstream tasks in AI4Research.<n>We identify key research gaps and highlight promising future directions.
  arXiv  Detail & Related papers  (2025-07-02T17:19:20Z)
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network [15.626884746513712]
  We use AI techniques to predict the future research directions of AI itself. For that, we use more than 100,000 research papers and build up a knowledge network with more than 64,000 concept nodes. The most powerful methods use a carefully curated set of network features, rather than an end-to-end AI approach.
  arXiv  Detail & Related papers  (2022-09-23T14:04:37Z)
• The application of artificial intelligence in software engineering: a review challenging conventional wisdom [0.9651131604396904]
  This survey chapter is a review of the most commonplace methods of AI applied to software engineering. The review covers methods between years 1975-2017, for the requirements phase, 46 major AI-driven methods are found. The purpose of this chapter is to answer the following questions: is there sufficient intelligence in the SE lifecycle?
  arXiv  Detail & Related papers  (2021-08-03T15:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.