Forecasting Future World Events with Neural Networks

• URL: http://arxiv.org/abs/2206.15474v1
• Date: Thu, 30 Jun 2022 17:59:14 GMT
• Title: Forecasting Future World Events with Neural Networks
• Authors: Andy Zou, Tristan Xiao, Ryan Jia, Joe Kwon, Mantas Mazeika, Richard Li, Dawn Song, Jacob Steinhardt, Owain Evans, Dan Hendrycks
• Abstract summary: Autocast is a dataset containing thousands of forecasting questions and an accompanying news corpus. The news corpus is organized by date, allowing us to precisely simulate the conditions under which humans made past forecasts. We test language models on our forecasting task and find that performance is far below a human expert baseline.
• Score: 68.43460909545063
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Forecasting future world events is a challenging but valuable task. Forecasts of climate, geopolitical conflict, pandemics and economic indicators help shape policy and decision making. In these domains, the judgment of expert humans contributes to the best forecasts. Given advances in language modeling, can these forecasts be automated? To this end, we introduce Autocast, a dataset containing thousands of forecasting questions and an accompanying news corpus. Questions are taken from forecasting tournaments, ensuring high quality, real-world importance, and diversity. The news corpus is organized by date, allowing us to precisely simulate the conditions under which humans made past forecasts (avoiding leakage from the future). Motivated by the difficulty of forecasting numbers across orders of magnitude (e.g. global cases of COVID-19 in 2022), we also curate IntervalQA, a dataset of numerical questions and metrics for calibration. We test language models on our forecasting task and find that performance is far below a human expert baseline. However, performance improves with increased model size and incorporation of relevant information from the news corpus. In sum, Autocast poses a novel challenge for large language models and improved performance could bring large practical benefits.

Related papers

• Super Resolution On Global Weather Forecasts [0.1747623282473278]
  Group seeks to improve upon existing deep learning based forecasting methods by increasing spatial resolutions of global weather predictions. Specifically, we are interested in performing super resolution (SR) on GraphCast temperature predictions by increasing the global precision from 1 degree of accuracy to 0.5 degrees.
  arXiv  Detail & Related papers  (2024-09-17T19:07:13Z)
• F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
  We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm. By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases. Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
  arXiv  Detail & Related papers  (2024-06-23T21:28:50Z)
• Can Language Models Use Forecasting Strategies? [14.332379032371612]
  We describe experiments using a novel dataset of real world events and associated human predictions. We find that models still struggle to make accurate predictions about the future.
  arXiv  Detail & Related papers  (2024-06-06T19:01:42Z)
• Forecasting the Future with Future Technologies: Advancements in Large Meteorological Models [3.332582598089642]
  The field of meteorological forecasting has undergone a significant transformation with the integration of large models. Models like FourCastNet, Pangu-Weather, GraphCast, ClimaX, and FengWu have made notable contributions by providing accurate, high-resolution forecasts.
  arXiv  Detail & Related papers  (2024-04-10T00:52:54Z)
• Approaching Human-Level Forecasting with Language Models [34.202996056121]
  We study whether language models (LMs) can forecast at the level of competitive human forecasters. We develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions.
  arXiv  Detail & Related papers  (2024-02-28T18:54:18Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting [56.73502043159699]
  This work presents FengWu-GHR, the first data-driven global weather forecasting model running at the 0.09$circ$ horizontal resolution. It introduces a novel approach that opens the door for operating ML-based high-resolution forecasts by inheriting prior knowledge from a low-resolution model. The hindcast of weather prediction in 2022 indicates that FengWu-GHR is superior to the IFS-HRES.
  arXiv  Detail & Related papers  (2024-01-28T13:23:25Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• GraphCast: Learning skillful medium-range global weather forecasting [107.40054095223779]
  We introduce a machine learning-based method called "GraphCast", which can be trained directly from reanalysis data. It predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute. We show that GraphCast significantly outperforms the most accurate operational deterministic systems on 90% of 1380 verification targets.
  arXiv  Detail & Related papers  (2022-12-24T18:15:39Z)
• ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data [43.400630267599084]
  Event forecasting is a challenging, yet important task, as humans seek to constantly plan for the future. We formulate a task, construct a dataset, and provide benchmarks for developing methods for event forecasting with large volumes of unstructured text data. We present our experiments on ForecastQA using BERT-based models and find that our best model achieves 60.1% accuracy on the dataset, which still lags behind human performance by about 19%.
  arXiv  Detail & Related papers  (2020-05-02T11:03:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.