GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks

• URL: http://arxiv.org/abs/2510.04374v1
• Date: Sun, 05 Oct 2025 21:36:43 GMT
• Title: GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks
• Abstract summary: GDPval is a benchmark evaluating AI capabilities on real-world economically valuable tasks.<n>GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP.<n>We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts.
• Score: 34.63352957472507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce GDPval, a benchmark evaluating AI model capabilities on real-world economically valuable tasks. GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP (Gross Domestic Product). Tasks are constructed from the representative work of industry professionals with an average of 14 years of experience. We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts. We also demonstrate that increased reasoning effort, increased task context, and increased scaffolding improves model performance on GDPval. Finally, we open-source a gold subset of 220 tasks and provide a public automated grading service at evals.openai.com to facilitate future research in understanding real-world model capabilities.

Related papers

• Atria Dawn: The Dawn of Agentic Superintelligence [237.73474618289063]
  We introduce Atria Dawn Preview, a foundation agentic language model designed for scientific research and engineering.<n>We examine the real research-and-development process behind this model as a case study of human--AI collaboration.
  arXiv  Detail & Related papers  (2026-09-14T16:22:30Z)
• StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [63.57092302641323]
  We introduce textbfStartupBench, an E2E agent benchmark grounded in market-validated AI startup products.<n>We study AI products with demonstrated adoption, together with their product and users, to identify real-world tasks for which AI has established practical demand.<n>Our results reveal that many market-validated agents remain beyond the reliable capabilities of current general-purpose agents.
  arXiv  Detail & Related papers  (2026-08-18T14:01:32Z)
• Economic Evaluations of Language Models [81.69183693497281]
  EconEvals is an open-source evaluation suite to measure capabilities relevant to tasks, work activities, and occupations in the US labor economy.<n>Our evaluations improve coverage over OpenAI's GDPval benchmark, which is the existing state-of-the-art that covers 5% of US occupations, at 500x lower cost.<n>Our estimates indicate that current models could save workers substantial time on at least half of their tasks in 47% of occupations.
  arXiv  Detail & Related papers  (2026-06-26T22:26:40Z)
• Agents' Last Exam [275.79071391137074]
  Agents' Last Exam (ALE) is a benchmark designed to evaluate AI agents on long horizon, economically valuable, real world tasks with verifiable outcomes.<n>Developed in collaboration with 250+ industry experts, ALE covers non-physical industries defined with reference to O*NET / SOC 2018.<n>Current results show that the hardest tier remains far from saturated: across mainstream harness and backbone configurations, the average full pass rate is below 1%.
  arXiv  Detail & Related papers  (2026-06-03T20:20:46Z)
• Forecasting Green Skill Demand in the Automotive Industry: Evidence from Online Job Postings [2.629180414452215]
  This study presents a computational framework to measure and predict green skill demand using online job postings from Mexico's automotive industry.<n>A two-stage pipeline combining multilingual embeddings and ESCO validation identifies 274 unique green skills across 8,576 occurrences.<n>Results show current demand is concentrated in operational sustainability practices, while the fastest-growing skills relate to renewable energy, recycling, and hydrogen technologies.
  arXiv  Detail & Related papers  (2026-05-06T16:51:53Z)
• BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows [12.076413424824679]
  Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional.<n>To evaluate frontier AI agents in a high-value, labor-intensive profession, we introduce BankerToolBench (BTB)<n>BTB is an open-source benchmark of end-to-end analytical deliverables routinely performed by junior investment bankers.
  arXiv  Detail & Related papers  (2026-04-13T11:02:32Z)
• FrontierFinance: A Long-Horizon Computer-Use Benchmark of Real-World Financial Tasks [4.444378412245679]
  We introduce FrontierFinance, a benchmark of 25 complex financial modeling tasks across five core finance models.<n>We engage human experts to define the tasks, create rubrics, grade LLMs, and perform the tasks themselves as human baselines.<n>We demonstrate that our human experts both receive higher scores on average, and are more likely to provide client-ready outputs than current state-of-the-art systems.
  arXiv  Detail & Related papers  (2026-04-07T14:15:45Z)
• How Well Does Agent Development Reflect Real-World Work? [89.17217057358285]
  We study the relationship between agent development efforts and the distribution of real-world human work by mapping benchmark instances to work domains and skills.<n>We reveal substantial mismatches between agent development that tends to be programming-centric, and the categories in which human labor and economic value are concentrated.
  arXiv  Detail & Related papers  (2026-03-01T17:55:49Z)
• WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models [114.95269118652163]
  We introduce WorldArena, a unified benchmark designed to evaluate embodied world models across both perceptual and functional dimensions.<n>WorldArena assesses models through three dimensions: video perception quality, measured with 16 metrics across six sub-dimensions; embodied task functionality, which evaluates world models as data engines, policy evaluators, and action planners integrating with subjective human evaluation.<n>Through extensive experiments on 14 representative models, we reveal a significant perception-functionality gap, showing that high visual quality does not necessarily translate into strong embodied task capability.
  arXiv  Detail & Related papers  (2026-02-09T18:09:20Z)
• Tendem: A Hybrid AI+Human Platform [3.9177738683910537]
  Tendem is a hybrid system where AI handles structured, repeatable work and Human Experts step in when the models fail or to verify results.<n>To assess Tendem's performance, we conducted a series of in-house evaluations on 94 real-world tasks.
  arXiv  Detail & Related papers  (2026-02-01T09:37:16Z)
• AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts [35.52607495764441]
  Large Language Models (LLMs) based autonomous agents demonstrate multifaceted capabilities to contribute substantially to economic production.<n>We introduce AgencyBench, a benchmark derived from daily AI usage, evaluating 6 core agentic capabilities across 32 real-world scenarios.<n>These scenarios require an average of 90 tool calls, 1 million tokens, and hours of execution time to resolve.
  arXiv  Detail & Related papers  (2026-01-16T07:22:20Z)
• Scaling Laws for Economic Productivity: Experimental Evidence in LLM-Assisted Consulting, Data Analyst, and Management Tasks [0.0]
  We find each year of AI model progress reduced task time by 8%.<n> productivity gains were significantly larger for non-agentic analytical tasks.<n>These findings suggest continued model scaling could boost U.S. productivity by approximately 20% over the next decade.
  arXiv  Detail & Related papers  (2025-12-24T18:24:29Z)
• UpBench: A Dynamically Evolving Real-World Labor-Market Agentic Benchmark Framework Built for Human-Centric AI [2.0619484032730813]
  UpBench is a benchmark grounded in real jobs drawn from the global Upwork labor marketplace.<n>Each task corresponds to a verified client transaction, anchoring evaluation in genuine work activity and financial outcomes.<n>UpBench employs a rubric-based evaluation framework, in which expert freelancers decompose each job into detailed, verifiable acceptance criteria and assess AI submissions with per-criterion feedback.
  arXiv  Detail & Related papers  (2025-11-15T17:39:37Z)
• The AI Productivity Index (APEX) [4.122962658725304]
  We introduce the first version of the AI Productivity Index (APEX), a benchmark for assessing whether frontier AI models can perform knowledge work with high economic value.<n>APEX-v1.0 contains 200 test cases and covers four domains: investment banking, management consulting, law, and primary medical care.<n>We evaluate 23 frontier models on APEX-v1.0 using an LM judge. GPT 5 (Thinking = High) achieves the highest mean score (64.2%), followed by Grok 4 (61.3%) and Gemini 2.5 Flash (Thinking = On) (60.4%)
  arXiv  Detail & Related papers  (2025-09-30T03:26:17Z)
• FieldWorkArena: Agentic AI Benchmark for Real Field Work Tasks [52.47895046206854]
  FieldWorkArena is a benchmark for agentic AI targeting real-world field work.<n>This paper defines a new action space that agentic AI should possess for real world work environment benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T08:21:46Z)
• Sustainability via LLM Right-sizing [21.17523328451591]
  Large language models (LLMs) have become increasingly embedded in organizational.<n>This study offers an empirical answer by evaluating eleven proprietary and open-weight LLMs across ten everyday occupational tasks.<n>Results show that GPT-4o delivers consistently superior performance but at a significantly higher cost and environmental footprint.
  arXiv  Detail & Related papers  (2025-04-17T04:00:40Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)
• Large Language Models at Work in China's Labor Market [3.9145097124275257]
  This paper explores the potential impacts of large language models (LLMs) on the Chinese labor market.<n>The results indicate a positive correlation between occupational exposure and both wage levels and experience premiums at the occupation level.<n>We then aggregate occupational exposure at the industry level to obtain industrial exposure scores.
  arXiv  Detail & Related papers  (2023-08-17T04:20:36Z)
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
  This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent. Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
  arXiv  Detail & Related papers  (2023-02-15T04:14:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.