ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions

• URL: http://arxiv.org/abs/2608.30517v1
• Date: Mon, 31 Aug 2026 09:52:29 GMT
• Title: ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions
• Abstract summary: ScienceArena is an olympiad-style benchmark from thirteen public science competitions in physics, chemistry, and biology.<n>We build ScienceArena through an expert-audited digitization pipeline that converts official exams, figures, solutions, and rubrics into structured items verified by olympiad medalists.<n> Medalist notes show that failures often stem from visual grounding, structure fidelity, and global problem control rather than missing terminology.
• Score: 20.614899115950156
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Benchmark saturation and data contamination increasingly obscure genuine scientific reasoning in frontier LLMs. We introduce \textsc{ScienceArena}, an olympiad-style benchmark from thirteen public science competitions in physics, chemistry, and biology, including IPhO and IChO 2025--2026, IBO 2023, USAPhO 2026, and USNCO 2025. Its open-ended, multi-step problems use process-credit rubrics, making faithful scoring difficult. We build ScienceArena through an expert-audited digitization pipeline that converts official exams, figures, solutions, and rubrics into structured items verified by olympiad medalists. To scale evaluation beyond costly human grading, we calibrate LLM-as-judge against medalist ground truth on archived answers from five models across IPhO and IChO; two strong judges stay within one point of expert total scores. Medalist notes show that failures often stem from visual grounding, structure fidelity, and global problem control rather than missing terminology. Evaluating fourteen recent LLMs with interleaved solving, we find that top models obtain medal-equivalent rubric scores on several public international exams, while chemistry and long-horizon consistency remain key bottlenecks. We provide an interactive \href{https://science-arena.onrender.com/}{demo}.

Related papers

• Post-Training Language Models for Gold-Medal Performance in Coding Competitions [70.8192632995446]
  We present an end-to-end specialization pipeline combining large-scale problem curation, synthetic reasoning traces, supervised fine-tuning, and reinforcement learning.<n>Using 22,000 curated problems, we train Nemotron-3-Nano-CC with SFT and RL and Nemotron-3-Ultra-CC (550B-A55B) with SFT alone.<n>On IOI 2025, Nano-CC improves from 130 points to 291 after post-training and to 468 with GenCorrect, exceeding the gold threshold of 438.3 while Ultra-CC reaches 502.
  arXiv  Detail & Related papers  (2026-09-02T17:33:37Z)
• Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling [108.48818215929494]
  We introduce a simple and unified recipe for converting a post-trained reasoning backbone into a rigorous olympiad-level solver.<n>We train a 30B-A3B backbone with SFT on around 340K sub-8K-token trajectories followed by 200 RL steps.<n>The resulting model, SU-01, supports stable reasoning on difficult problems with trajectories exceeding 100K tokens.
  arXiv  Detail & Related papers  (2026-05-13T10:13:26Z)
• FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks [3.596967766594675]
  We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models.<n>For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task.
  arXiv  Detail & Related papers  (2026-01-29T02:04:56Z)
• Large Language Models Achieve Gold Medal Performance at the International Olympiad on Astronomy & Astrophysics (IOAA) [43.53870250026015]
  We benchmark five large language models (LLMs) on the International Olympiad on Astronomy and Astrophysics (IOAA) exams.<n>With average scores of 85.6% and 84.2%, Gemini 2.5 Pro and GPT-5 rank in the top two among 200-300 participants in all four IOAA theory exams evaluated.<n>GPT-5 still excels in the exams with an 88.5% average score, ranking top 10 among the participants in the four most recent IOAAs.
  arXiv  Detail & Related papers  (2025-10-06T16:58:47Z)
• PhysicsMinions: Winning Gold Medals in the Latest Physics Olympiads with a Coevolutionary Multimodal Multi-Agent System [65.02248709992442]
  Physics is central to understanding and shaping the real world, and the ability to solve physics problems is a key indicator of real-world physical intelligence.<n>Existing approaches are predominantly single-model based, and open-source MLLMs rarely reach gold-medal-level performance.<n>We propose PhysicsMinions, a coevolutionary multi-agent system for Physics Olympiad.<n>Its architecture features three synergistic studios: a Visual Studio to interpret diagrams, a Logic Studio to formulate solutions, and a Review Studio to perform dual-stage verification.
  arXiv  Detail & Related papers  (2025-09-29T14:40:53Z)
• HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark? [53.76627321546095]
  HiPhO is the first benchmark dedicated to high school physics Olympiads with human-aligned evaluation.<n>It compiles 13 latest Olympiad exams from 2024-2025, spanning both international and regional competitions.<n>We assign gold, silver, and bronze medals to models based on official medal thresholds, thereby enabling direct comparison between (M)LLMs and human contestants.
  arXiv  Detail & Related papers  (2025-09-09T16:24:51Z)
• LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? [88.29001498765629]
  Large language models (LLMs) now outperform elite humans in competitive programming.<n>We revisit this claim, examining how LLMs differ from human experts and where limitations still remain.<n>We introduce LiveCodeBench Pro, a benchmark composed of problems from Codeforces, ICPC, and IOI.<n>A team of Olympiad medalists annotates every problem for algorithmic categories and conducts a line-by-line analysis of failed model-generated submissions.
  arXiv  Detail & Related papers  (2025-06-13T16:29:09Z)
• OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems [62.06169250463104]
  We present OlympiadBench, an Olympiad-level bilingual multimodal scientific benchmark, featuring 8,476 problems from Olympiad-level mathematics and physics competitions. The best-performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics. Our analysis orienting GPT-4V points out prevalent issues with hallucinations, knowledge omissions, and logical fallacies.
  arXiv  Detail & Related papers  (2024-02-21T18:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.