Hybrid Forecasting of Geopolitical Events

• URL: http://arxiv.org/abs/2412.10981v1
• Date: Sat, 14 Dec 2024 22:09:45 GMT
• Title: Hybrid Forecasting of Geopolitical Events
• Authors: Daniel M. Benjamin, Fred Morstatter, Ali E. Abbas, Andres Abeliuk, Pavel Atanasov, Stephen Bennett, Andreas Beger, Saurabh Birari, David V. Budescu, Michele Catasta, Emilio Ferrara, Lucas Haravitch, Mark Himmelstein, KSM Tozammel Hossain, Yuzhong Huang, Woojeong Jin, Regina Joseph, Jure Leskovec, Akira Matsui, Mehrnoosh Mirtaheri, Xiang Ren, Gleb Satyukov, Rajiv Sethi, Amandeep Singh, Rok Sosic, Mark Steyvers, Pedro A Szekely, Michael D. Ward, Aram Galstyan,
• Abstract summary: SAGE is a hybrid forecasting system that combines human and machine generated forecasts. The system aggregates human and machine forecasts weighting both for propinquity and based on assessed skill. We show that skilled forecasters who had access to machine-generated forecasts outperformed those who only viewed historical data.
• Score: 71.73737011120103
• License:
• Abstract: Sound decision-making relies on accurate prediction for tangible outcomes ranging from military conflict to disease outbreaks. To improve crowdsourced forecasting accuracy, we developed SAGE, a hybrid forecasting system that combines human and machine generated forecasts. The system provides a platform where users can interact with machine models and thus anchor their judgments on an objective benchmark. The system also aggregates human and machine forecasts weighting both for propinquity and based on assessed skill while adjusting for overconfidence. We present results from the Hybrid Forecasting Competition (HFC) - larger than comparable forecasting tournaments - including 1085 users forecasting 398 real-world forecasting problems over eight months. Our main result is that the hybrid system generated more accurate forecasts compared to a human-only baseline which had no machine generated predictions. We found that skilled forecasters who had access to machine-generated forecasts outperformed those who only viewed historical data. We also demonstrated the inclusion of machine-generated forecasts in our aggregation algorithms improved performance, both in terms of accuracy and scalability. This suggests that hybrid forecasting systems, which potentially require fewer human resources, can be a viable approach for maintaining a competitive level of accuracy over a larger number of forecasting questions.

Related papers

• Consistency Checks for Language Model Forecasters [54.62507816753479]
  We measure the performance of forecasters in terms of the consistency of their predictions on different logically-related questions. We build an automated evaluation system that generates a set of base questions, instantiates consistency checks from these questions, elicits predictions of the forecaster, and measures the consistency of the predictions.
  arXiv  Detail & Related papers  (2024-12-24T16:51:35Z)
• Performative Prediction on Games and Mechanism Design [69.7933059664256]
  We study a collective risk dilemma where agents decide whether to trust predictions based on past accuracy. As predictions shape collective outcomes, social welfare arises naturally as a metric of concern. We show how to achieve better trade-offs and use them for mechanism design.
  arXiv  Detail & Related papers  (2024-08-09T16:03:44Z)
• Approaching Human-Level Forecasting with Language Models [34.202996056121]
  We study whether language models (LMs) can forecast at the level of competitive human forecasters. We develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions.
  arXiv  Detail & Related papers  (2024-02-28T18:54:18Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Beyond S-curves: Recurrent Neural Networks for Technology Forecasting [60.82125150951035]
  We develop an autencoder approach that employs recent advances in machine learning and time series forecasting. S-curves forecasts largely exhibit a mean average percentage error (MAPE) comparable to a simple ARIMA baseline. Our autoencoder approach improves the MAPE by 13.5% on average over the second-best result.
  arXiv  Detail & Related papers  (2022-11-28T14:16:22Z)
• A two-step machine learning approach to statistical post-processing of weather forecasts for power generation [0.0]
  Wind and solar energy sources are highly volatile making planning difficult for grid operators. We propose a two-step machine learning-based approach to calibrating ensemble weather forecasts.
  arXiv  Detail & Related papers  (2022-07-15T16:38:14Z)
• Random Forest of Epidemiological Models for Influenza Forecasting [7.050453841068465]
  We propose a Tree Ensemble model design that utilizes the individual predictors of our baseline model SIkJalpha to improve its performance. We demonstrate that our Random Forest-based approach is able to improve upon the forecasts of the individual predictors in terms of mean absolute error, coverage, and weighted interval score.
  arXiv  Detail & Related papers  (2022-06-17T18:47:40Z)
• Computing the ensemble spread from deterministic weather predictions using conditional generative adversarial networks [0.0]
  We propose to use deep-learning algorithms to learn the statistical properties of an ensemble prediction system. Once trained, the costly ensemble prediction system will not be needed anymore to obtain future ensemble forecasts.
  arXiv  Detail & Related papers  (2022-05-18T19:10:38Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Calibration of wind speed ensemble forecasts for power generation [0.0]
  In the last decades wind power became the second largest energy source in the EU covering 16% of its electricity demand. Due to its volatility, accurate short range wind power predictions are required for successful integration of wind energy into the electrical grid. We show that compared with the raw ensemble, post-processing always improves the calibration of probabilistic and accuracy of point forecasts.
  arXiv  Detail & Related papers  (2021-04-30T11:18:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.