Asking Better Questions -- The Art and Science of Forecasting: A mechanism for truer answers to high-stakes questions

• URL: http://arxiv.org/abs/2303.18006v1
• Date: Fri, 31 Mar 2023 12:27:26 GMT
• Title: Asking Better Questions -- The Art and Science of Forecasting: A mechanism for truer answers to high-stakes questions
• Authors: Emily Dardaman (1) and Abhishek Gupta (1, 2, and 3) ((1) BCG Henderson Institute, (2) Montreal AI Ethics Institute, (3) Boston Consulting Group)
• Abstract summary: Without the ability to estimate and benchmark AI capability advancements, organizations are left to respond to each change reactively. This paper explores the recent growth of forecasting, a political science tool that uses explicit assumptions and quantitative estimation. Forecasting can identify and verify talent, enable leaders to build better models of AI advancements and improve inputs into design policy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Without the ability to estimate and benchmark AI capability advancements, organizations are left to respond to each change reactively, impeding their ability to build viable mid and long-term strategies. This paper explores the recent growth of forecasting, a political science tool that uses explicit assumptions and quantitative estimation that leads to improved prediction accuracy. Done at the collective level, forecasting can identify and verify talent, enable leaders to build better models of AI advancements and improve inputs into design policy. Successful approaches to forecasting and case studies are examined, revealing a subclass of "superforecasters" who outperform 98% of the population and whose insights will be most reliable. Finally, techniques behind successful forecasting are outlined, including Phillip Tetlock's "Ten Commandments." To adapt to a quickly changing technology landscape, designers and policymakers should consider forecasting as a first line of defense.

Related papers

• Performative Prediction on Games and Mechanism Design [69.7933059664256]
  We study a collective risk dilemma where agents decide whether to trust predictions based on past accuracy. As predictions shape collective outcomes, social welfare arises naturally as a metric of concern. We show how to achieve better trade-offs and use them for mechanism design.
  arXiv  Detail & Related papers  (2024-08-09T16:03:44Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI. It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• Designing Decision Support Systems Using Counterfactual Prediction Sets [15.121082690769525]
  Decision support systems for classification tasks are predominantly designed to predict the value of the ground truth labels. This paper revisits the design of this type of systems from the perspective of online learning. We develop a methodology that does not require, nor assumes, an expert model.
  arXiv  Detail & Related papers  (2023-06-06T18:00:09Z)
• Travel Demand Forecasting: A Fair AI Approach [0.9383397937755517]
  We propose a novel methodology to develop fairness-aware, highly-accurate travel demand forecasting models. Specifically, we introduce a new fairness regularization term, which is explicitly designed to measure the correlation between prediction accuracy and protected attributes. Results highlight that our proposed methodology can effectively enhance fairness for multiple protected attributes while preserving prediction accuracy.
  arXiv  Detail & Related papers  (2023-03-03T03:16:54Z)
• Forecasting Future World Events with Neural Networks [68.43460909545063]
  Autocast is a dataset containing thousands of forecasting questions and an accompanying news corpus. The news corpus is organized by date, allowing us to precisely simulate the conditions under which humans made past forecasts. We test language models on our forecasting task and find that performance is far below a human expert baseline.
  arXiv  Detail & Related papers  (2022-06-30T17:59:14Z)
• What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience [63.75363908696257]
  computational reinforcement learning seeks to construct an agent's perception of the world through predictions of future sensations. An open challenge in this line of work is determining from the infinitely many predictions that the agent could possibly make which predictions might best support decision-making. We introduce a meta-gradient descent process by which an agent learns what predictions to make, 2) the estimates for its chosen predictions, and 3) how to use those estimates to generate policies that maximize future reward.
  arXiv  Detail & Related papers  (2022-06-13T21:31:06Z)
• Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty [54.88405167739227]
  We present HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities. We additionally present PUP, a new challenging real-world autonomous driving dataset. We demonstrate that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty.
  arXiv  Detail & Related papers  (2021-04-26T10:28:34Z)
• Forecasting AI Progress: A Research Agenda [0.41998444721319206]
  This paper describes the development of a research agenda for forecasting AI progress. It uses the Delphi technique to elicit and aggregate experts' opinions on what questions and methods to prioritize. Experts indicated that a wide variety of methods should be considered for forecasting AI progress.
  arXiv  Detail & Related papers  (2020-08-04T21:46:46Z)
• Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation [65.92151529708036]
  We introduce greedy policy search (GPS) as a simple but high-performing method for learning a policy of test-time augmentation. We demonstrate that augmentation policies learned with GPS achieve superior predictive performance on image classification problems.
  arXiv  Detail & Related papers  (2020-02-21T02:57:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.