The Open Source Economic Index of AI Adoption and Capability

• URL: http://arxiv.org/abs/2606.26118v1
• Date: Sat, 23 May 2026 18:45:42 GMT
• Title: The Open Source Economic Index of AI Adoption and Capability
• Abstract summary: We work towards measuring both AI adoption and the capability of AI to perform discrete labor tasks across various occupations.<n>We develop an open-source economic index that uses publicly available user-LLM chat data and O*NET tasks to replicate studies produced by frontier AI labs.<n>We build a system that generates benchmark scenarios grounded in O*NET occupations, tasks, and model-context-protocol (MCP) servers.
• Score: 27.900759713490576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We work towards measuring both AI adoption and the capability of AI to perform discrete labor tasks across various occupations. To measure adoption, we develop an open-source economic index that uses publicly available user-LLM chat data and O*NET tasks to replicate studies produced by frontier AI labs, finding that occupations in the finance, computer science, and arts sectors are those with the highest adoption rates. To measure capabilities, we build a system that generates benchmark scenarios grounded in O*NET occupations, tasks, and model-context-protocol (MCP) servers. We test Kimi-k2.5 with an OpenAI agents SDK harness on scenarios across 9 occupations that appear frequently in our index, finding that AI correctly executes high-level workflows but often errs in the granular details (such as specific tool calls used).

Related papers

• When Does AI Augment Work? A Workflow-Level Framework for Human-Agent Collaboration [78.16115745870405]
  We aim to characterise the value of artificial intelligence in the workplace.<n>Current studies largely measure this value in terms of the current automation capabilities and public adoption of AI.<n>To account for this, we must expand the scope of our analysis beyond atomised tasks of today.
  arXiv  Detail & Related papers  (2026-09-11T06:29:34Z)
• Using profiles of cognitive capability to assess AI suitability for workplace tasks [5.826026042207654]
  We introduce a pipeline that profiles agents and tasks using a shared set of core cognitive capabilities.<n>As both use a common set of cognitive dimensions, they can be updated independently as models and roles change.<n>We validate capability recovery on synthetic agents, profile six AI systems, and elicit task requirements from 410 employees.
  arXiv  Detail & Related papers  (2026-08-26T10:48:09Z)
• StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [63.57092302641323]
  We introduce textbfStartupBench, an E2E agent benchmark grounded in market-validated AI startup products.<n>We study AI products with demonstrated adoption, together with their product and users, to identify real-world tasks for which AI has established practical demand.<n>Our results reveal that many market-validated agents remain beyond the reliable capabilities of current general-purpose agents.
  arXiv  Detail & Related papers  (2026-08-18T14:01:32Z)
• Where can AI be used? Insights from a deep ontology of work activities [14.450099263467926]
  Artificial intelligence (AI) is poised to profoundly reshape how work is executed and organized.<n>We provide a comprehensive ontology of work activities that can help systematically analyze and predict uses of AI.<n>We use this framework to classify descriptions of 13,275 AI software applications and a worldwide tally of 20.8 million robotic systems.
  arXiv  Detail & Related papers  (2026-03-21T03:28:39Z)
• On the Carbon Footprint of Economic Research in the Age of Generative AI [0.0]
  Green AI research has largely measured the footprint of models rather than the downstream in which GenAI is a tool.<n>We shift the iteration unit of analysis from models to topic and treat prompts as decision policies that allocate discretion between researcher and system.<n>Results identify human in the loop governance as a practical lever to align GenAI productivity with environmental efficiency.
  arXiv  Detail & Related papers  (2026-03-17T22:18:32Z)
• Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Search [55.96277616578607]
  We formalize the problem of Human-centric Open-future Task Discovery (HOTD), focusing on identifying tasks that reduce human effort across multiple plausible futures.<n>To facilitate this study, we propose an HOTD-Bench, which features over 2K real-world videos, a semi-automated annotation pipeline, and a simulation-based protocol tailored for open-set future evaluation.<n>We also propose the Collaborative Multi-Agent Search Tree (CMAST), which decomposes the complex reasoning through a multi-agent system and structures the reasoning process through a scalable search tree module.
  arXiv  Detail & Related papers  (2025-11-24T09:33:59Z)
• Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations [18.86239369192921]
  We leverage a recent privacy-preserving system to analyze over four million Claude.ai conversations.<n>Our analysis reveals that AI usage primarily concentrates in software development and writing tasks.<n>We also analyze how AI is being used for tasks, finding 57% of usage suggests augmentation of human capabilities.
  arXiv  Detail & Related papers  (2025-02-11T00:46:43Z)
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [55.03911355902567]
  We introduce TheAgentCompany, a benchmark for evaluating AI agents that interact with the world in similar ways to those of a digital worker.<n>We find that the most competitive agent can complete 30% of tasks autonomously.<n>This paints a nuanced picture on task automation with simulating LM agents in a setting a real workplace.
  arXiv  Detail & Related papers  (2024-12-18T18:55:40Z)
• Towards the Terminator Economy: Assessing Job Exposure to AI through LLMs [10.844598404826355]
  One-third of U.S. employment is highly exposed to AI, primarily in high-skill jobs requiring a graduate or postgraduate level of education.<n>Even in high-skill occupations, AI exhibits high variability in task substitution, suggesting that AI and humans complement each other within the same occupation.<n>All results, models, and code are freely available online to allow the community to reproduce our results, compare outcomes, and use our work as a benchmark to monitor AI's progress over time.
  arXiv  Detail & Related papers  (2024-07-27T08:14:18Z)
• WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks [85.95607119635102]
  Large language models (LLMs) can mimic human-like intelligence.<n>WorkArena++ is designed to evaluate the planning, problem-solving, logical/arithmetic reasoning, retrieval, and contextual understanding abilities of web agents.
  arXiv  Detail & Related papers  (2024-07-07T07:15:49Z)
• WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? [83.19032025950986]
  We study the use of large language model-based agents for interacting with software via web browsers. WorkArena is a benchmark of 33 tasks based on the widely-used ServiceNow platform. BrowserGym is an environment for the design and evaluation of such agents.
  arXiv  Detail & Related papers  (2024-03-12T14:58:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.