FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks

• URL: http://arxiv.org/abs/2601.21165v1
• Date: Thu, 29 Jan 2026 02:04:56 GMT
• Title: FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks
• Abstract summary: We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models.<n>For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task.
• Score: 3.596967766594675
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models. Recent model progress has nearly saturated existing science benchmarks, which often rely on multiple-choice knowledge questions or already published information. FrontierScience addresses this gap through two complementary tracks: (1) Olympiad, consisting of international olympiad problems at the level of IPhO, IChO, and IBO, and (2) Research, consisting of PhD-level, open-ended problems representative of sub-tasks in scientific research. FrontierScience contains several hundred questions (including 160 in the open-sourced gold set) covering subfields across physics, chemistry, and biology, from quantum electrodynamics to synthetic organic chemistry. All Olympiad problems are originally produced by international Olympiad medalists and national team coaches to ensure standards of difficulty, originality, and factuality. All Research problems are research sub-tasks written and verified by PhD scientists (doctoral candidates, postdoctoral researchers, or professors). For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task, rather than judging only a standalone final answer.

Related papers

• ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions [20.614899115950156]
  ScienceArena is an olympiad-style benchmark from thirteen public science competitions in physics, chemistry, and biology.<n>We build ScienceArena through an expert-audited digitization pipeline that converts official exams, figures, solutions, and rubrics into structured items verified by olympiad medalists.<n> Medalist notes show that failures often stem from visual grounding, structure fidelity, and global problem control rather than missing terminology.
  arXiv  Detail & Related papers  (2026-08-31T09:52:29Z)
• PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research [43.71141859083647]
  PRL-Bench is a curated benchmark for executing end-to-end physics research.<n>It covers astrophysics, condensed matter physics, high-energy physics, quantum information, and statistical physics.<n>Each task in the benchmark is designed to replicate the core properties of authentic scientific research.
  arXiv  Detail & Related papers  (2026-04-16T16:22:04Z)
• BABE: Biology Arena BEnchmark [51.53220868983288]
  BABE is a benchmark designed to evaluate the experimental reasoning capabilities of biological AI systems.<n>Our benchmark provides a robust framework for assessing how well AI systems can reason like practicing scientists.
  arXiv  Detail & Related papers  (2026-02-05T16:39:20Z)
• Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows [203.3527268311731]
  We present an operational SGI definition grounded in the Practical Inquiry Model (PIM)<n>We operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning.<n>Our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
  arXiv  Detail & Related papers  (2025-12-18T12:44:36Z)
• ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning [118.46980291324148]
  ATLAS is a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems.<n>Its key features include: High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage.<n>Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities.
  arXiv  Detail & Related papers  (2025-11-18T11:13:06Z)
• Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark [49.42250115889234]
  We present the first benchmark designed to test large language models (LLMs) on research-level reasoning tasks.<n>CritPt consists of 71 composite research challenges designed to simulate full-scale research projects at the entry level.<n>We find that while current state-of-the-art LLMs show early promise on isolated checkpoints, they remain far from being able to reliably solve full research-scale challenges.
  arXiv  Detail & Related papers  (2025-09-30T17:34:03Z)
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning [53.82037883518254]
  We introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks.<n>We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks.
  arXiv  Detail & Related papers  (2025-08-26T17:04:23Z)
• CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning [12.396302011805755]
  We introduce CURIE, a benchmark to measure the potential of Large Language Models (LLMs) in scientific problem-solving.<n>This benchmark introduces ten challenging tasks with a total of 580 problems and solution pairs curated by experts in six disciplines.<n>We evaluate a range of closed and open LLMs on tasks in CURIE which requires domain expertise, comprehension of long in-context information,and multi-step reasoning.
  arXiv  Detail & Related papers  (2025-03-14T17:53:03Z)
• OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems [62.06169250463104]
  We present OlympiadBench, an Olympiad-level bilingual multimodal scientific benchmark, featuring 8,476 problems from Olympiad-level mathematics and physics competitions. The best-performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics. Our analysis orienting GPT-4V points out prevalent issues with hallucinations, knowledge omissions, and logical fallacies.
  arXiv  Detail & Related papers  (2024-02-21T18:49:26Z)
• Scientific Large Language Models: A Survey on Biological & Chemical Domains [47.97810890521825]
  Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration.
  arXiv  Detail & Related papers  (2024-01-26T05:33:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.