Jobs' AI Exposure Should Be Measured from Evidence, Not Model Priors

• URL: http://arxiv.org/abs/2605.15474v1
• Date: Thu, 14 May 2026 23:29:42 GMT
• Title: Jobs' AI Exposure Should Be Measured from Evidence, Not Model Priors
• Abstract summary: Job exposure to AI should be measured with grounded, evidence-based methods.<n>Current theoretical exposure measures use zero-shot prompting to classify task-level AI exposure.<n>We propose a retrieval-augmented framework that assigns AI exposure labels to all 18,796 occupation--task pairs in O*NET 30.2.
• Score: 2.4724375110574304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This position paper argues that job exposure to AI should be measured with grounded, evidence-based methods, not inferred from LLM priors alone. Current theoretical exposure measures use zero-shot prompting to classify task-level AI exposure, generating labels with no explicit evidence, no transparent chain of reasoning, and no external validation. The stakes of these measurements are too high to rely on such methods, as they influence policy making, where public and private funds are directed, and how workers understand their future prospects. We therefore argue that AI capability claims should meet three standards: reproducibility, external grounding, and inspectability. We propose a retrieval-augmented framework that assigns AI exposure labels to all 18,796 occupation--task pairs in O*NET 30.2, using open-weight reasoning and instruct models with retrieved news articles and academic paper abstracts as evidence of current AI capabilities. Relative to a zero-shot baseline, the grounded condition is preferred in over 72\% of disagreement cases under both automatic and human evaluation, and yields scores that align more closely with observed real-world AI usage. Taken together, these findings suggest that evidence-grounded measurement better captures what current AI systems can plausibly do in practice, rather than what a model asserts without external evidence. Because AI capabilities continue to change, the measurements used to inform policy must evolve with them: theoretical AI exposure scores should be periodically reassessed, not inherited as immutable ground truth.

Related papers

• What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Education [3.2940515388832963]
  This paper develops a framework for AI-mediated assessment that links the learning claim, delegation boundary, evidence standard, and safeguards.<n>We then audit verified public generative AI assessment guidance from 30 universities.
  arXiv  Detail & Related papers  (2026-07-22T10:21:16Z)
• VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents [57.240036084348354]
  We introduce VASO, a framework for verification-guided self-evolution of robot skill contracts.<n>VASO verifies logically inconsistent skill contracts, then verifies plans induced by the skill against global and local temporal specifications.<n>On Clearpath Jackal and PX4 quadcopter tasks, VASO reaches 97.2% formal-specification compliance using fewer than 100 optimization samples.
  arXiv  Detail & Related papers  (2026-06-03T20:02:35Z)
• Trustworthy AI Suffers from Invariance Conflicts and Causality is The Solution [80.98492754957466]
  Core trustworthy AI objectives, such as fairness, robustness, privacy, and explainability, are hard to achieve simultaneously.<n>This paper argues that causality is necessary to understand and balance trade-offs in performance and multiple objectives of trustworthy AI.
  arXiv  Detail & Related papers  (2026-05-04T14:26:28Z)
• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)
• Questionnaire Responses Do not Capture the Safety of AI Agents [0.0]
  A fast-growing field of AI research is devoted to developing such assessments.<n>Standard methods prompt large language models (LLMs) in a questionnaire-style to describe their values or behavior in hypothetical scenarios.<n>We argue that a structurally identical issue holds for current AI alignment approaches.
  arXiv  Detail & Related papers  (2026-03-15T15:01:09Z)
• Measuring What AI Systems Might Do: Towards A Measurement Science in AI [19.687397197326817]
  capabilities, propensities, skills, values, and abilities are routinely used interchangeably and conflated with observable performance.<n>We argue that capabilities and propensities are dispositional properties - stable features of systems characterised by counterfactual relationships.
  arXiv  Detail & Related papers  (2026-02-10T12:59:41Z)
• Rethinking Data Protection in the (Generative) Artificial Intelligence Era [138.07763415496288]
  We propose a four-level taxonomy that captures the diverse protection needs arising in modern (generative) AI models and systems.<n>Our framework offers a structured understanding of the trade-offs between data utility and control, spanning the entire AI pipeline.
  arXiv  Detail & Related papers  (2025-07-03T02:45:51Z)
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• AI exposure predicts unemployment risk [1.5101132008238312]
  We assess which models of AI exposure predict job separations and unemployment risk. We find that individual AI exposure models are not predictive of unemployment rates, unemployment risk, or job separation rates. Our results also call for dynamic, context-aware, and validated methods for assessing AI exposure.
  arXiv  Detail & Related papers  (2023-08-04T15:21:07Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Should Machine Learning Models Report to Us When They Are Clueless? [0.0]
  We report that AI models extrapolate outside their range of familiar data. Knowing whether a model has extrapolated or not is a fundamental insight that should be included in explaining AI models.
  arXiv  Detail & Related papers  (2022-03-23T01:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.