Forecasting AI Progress: A Research Agenda
- URL: http://arxiv.org/abs/2008.01848v1
- Date: Tue, 4 Aug 2020 21:46:46 GMT
- Title: Forecasting AI Progress: A Research Agenda
- Authors: Ross Gruetzemacher, Florian Dorner, Niko Bernaola-Alvarez, Charlie
Giattino, David Manheim
- Abstract summary: This paper describes the development of a research agenda for forecasting AI progress.
It uses the Delphi technique to elicit and aggregate experts' opinions on what questions and methods to prioritize.
Experts indicated that a wide variety of methods should be considered for forecasting AI progress.
- Score: 0.41998444721319206
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Forecasting AI progress is essential to reducing uncertainty in order to
appropriately plan for research efforts on AI safety and AI governance. While
this is generally considered to be an important topic, little work has been
conducted on it and there is no published document that gives and objective
overview of the field. Moreover, the field is very diverse and there is no
published consensus regarding its direction. This paper describes the
development of a research agenda for forecasting AI progress which utilized the
Delphi technique to elicit and aggregate experts' opinions on what questions
and methods to prioritize. The results of the Delphi are presented; the
remainder of the paper follow the structure of these results, briefly reviewing
relevant literature and suggesting future work for each topic. Experts
indicated that a wide variety of methods should be considered for forecasting
AI progress. Moreover, experts identified salient questions that were both
general and completely unique to the problem of forecasting AI progress. Some
of the highest priority topics include the validation of (partially unresolved)
forecasts, how to make forecasting action-guiding and the quality of different
performance metrics. While statistical methods seem more promising, there is
also recognition that supplementing judgmental techniques can be quite
beneficial.
Related papers
- Open Problems in Technical AI Governance [93.89102632003996]
Technical AI governance refers to technical analysis and tools for supporting the effective governance of AI.
This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.
arXiv Detail & Related papers (2024-07-20T21:13:56Z) - Systematic literature review on forecasting and prediction of technical debt evolution [0.0]
Technical debt (TD) refers to the additional costs incurred due to compromises in software quality.
This study aims to explore existing knowledge in software engineering to gain insights into approaches proposed in research and industry.
arXiv Detail & Related papers (2024-06-17T18:50:37Z) - A Systematic Literature Review on Explainability for Machine/Deep
Learning-based Software Engineering Research [23.966640472958105]
This paper presents a systematic literature review of approaches that aim to improve the explainability of AI models within the context of Software Engineering.
We aim to summarize the SE tasks where XAI techniques have shown success to date; (2) classify and analyze different XAI techniques; and (3) investigate existing evaluation approaches.
arXiv Detail & Related papers (2024-01-26T03:20:40Z) - Explainable Authorship Identification in Cultural Heritage Applications:
Analysis of a New Perspective [48.031678295495574]
We explore the applicability of existing general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId.
In particular, we assess the relative merits of three different types of XAI techniques on three different AId tasks.
Our analysis shows that, while these techniques make important first steps towards explainable Authorship Identification, more work remains to be done.
arXiv Detail & Related papers (2023-11-03T20:51:15Z) - Predictable Artificial Intelligence [77.1127726638209]
This paper introduces the ideas and challenges of Predictable AI.
It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems.
We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
arXiv Detail & Related papers (2023-10-09T21:36:21Z) - Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions [2.35574869517894]
This study focuses on analyzing the recent advances in the area of Multimodal XAI (MXAI)
MXAI comprises methods that involve multiple modalities in the primary prediction and explanation tasks.
arXiv Detail & Related papers (2023-06-09T07:51:50Z) - Asking Better Questions -- The Art and Science of Forecasting: A
mechanism for truer answers to high-stakes questions [0.0]
Without the ability to estimate and benchmark AI capability advancements, organizations are left to respond to each change reactively.
This paper explores the recent growth of forecasting, a political science tool that uses explicit assumptions and quantitative estimation.
Forecasting can identify and verify talent, enable leaders to build better models of AI advancements and improve inputs into design policy.
arXiv Detail & Related papers (2023-03-31T12:27:26Z) - The Role of AI in Drug Discovery: Challenges, Opportunities, and
Strategies [97.5153823429076]
The benefits, challenges and drawbacks of AI in this field are reviewed.
The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
arXiv Detail & Related papers (2022-12-08T23:23:39Z) - Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research.
An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system.
We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
arXiv Detail & Related papers (2022-04-11T14:36:39Z) - An interdisciplinary conceptual study of Artificial Intelligence (AI)
for helping benefit-risk assessment practices: Towards a comprehensive
qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence.
The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
arXiv Detail & Related papers (2021-05-07T12:01:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.