Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling

• URL: http://arxiv.org/abs/2411.14042v1
• Date: Thu, 21 Nov 2024 11:44:23 GMT
• Title: Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling
• Authors: Daehoon Gwak, Junwoo Park, Minho Park, Chaehun Park, Hyunchan Lee, Edward Choi, Jaegul Choo,
• Abstract summary: WorldREP is a novel dataset designed to address limitations by leveraging the advanced reasoning capabilities of large-language models (LLMs) Our dataset features high-quality scoring labels generated through advanced prompt modeling and rigorously validated by domain experts in political science. We publicly release our dataset along with the full automation source code for data collection, labeling, and benchmarking, aiming to support and advance research in text-based event prediction.
• Score: 37.508538729757404
• License:
• Abstract: Predicting future international events from textual information, such as news articles, has tremendous potential for applications in global policy, strategic decision-making, and geopolitics. However, existing datasets available for this task are often limited in quality, hindering the progress of related research. In this paper, we introduce WORLDREP (WORLD Relationship and Event Prediction), a novel dataset designed to address these limitations by leveraging the advanced reasoning capabilities of large-language models (LLMs). Our dataset features high-quality scoring labels generated through advanced prompt modeling and rigorously validated by domain experts in political science. We showcase the quality and utility of WORLDREP for real-world event prediction tasks, demonstrating its effectiveness through extensive experiments and analysis. Furthermore, we publicly release our dataset along with the full automation source code for data collection, labeling, and benchmarking, aiming to support and advance research in text-based event prediction.

Related papers

• Predicting Country Instability Using Bayesian Deep Learning and Random Forest [0.0]
  Country instability is a global issue, with unpredictably high levels of instability thwarting socio-economic growth and possibly causing a slew of negative consequences. The Global Database of Activities, Voice, and Tone (GDELT Project) records broadcast, print, and web news in over 100 languages every second of every day. The main goal of our research is to investigate how, when our data grows more voluminous and fine-grained, we can conduct a more complex methodological analysis of political conflict.
  arXiv  Detail & Related papers  (2024-11-11T00:23:03Z)
• Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
  "Context is Key" (CiK) is a time series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings.
  arXiv  Detail & Related papers  (2024-10-24T17:56:08Z)
• MIRAI: Evaluating LLM Agents for Event Forecasting [22.524158637977]
  We introduce MIRAI, a novel benchmark designed to evaluate LLM agents as temporal forecasters in the context of international events. Our benchmark features an agentic environment with tools for accessing an extensive database of historical, structured events and textual news articles. In summary, MIRAI comprehensively evaluates the agents' capabilities in three dimensions: 1) autonomously source and integrate critical information from large global databases; 2) write codes using domain-specific APIs and libraries for tool-use; and 3) jointly reason over historical knowledge from diverse formats and time to accurately predict future events.
  arXiv  Detail & Related papers  (2024-07-01T12:22:46Z)
• Data-Centric AI in the Age of Large Language Models [51.20451986068925]
  This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs) We make the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs. We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization.
  arXiv  Detail & Related papers  (2024-06-20T16:34:07Z)
• SCTc-TE: A Comprehensive Formulation and Benchmark for Temporal Event Forecasting [63.01035584154509]
  We develop a fully automated pipeline and construct a large-scale dataset named MidEast-TE from about 0.6 million news articles. This dataset focuses on the cooperation and conflict events among countries mainly in the MidEast region from 2015 to 2022. We propose a novel method LoGo that is able to take advantage of both Local and Global contexts for SCTc-TE forecasting.
  arXiv  Detail & Related papers  (2023-12-02T07:40:21Z)
• Exploring the Potential of AI-Generated Synthetic Datasets: A Case Study on Telematics Data with ChatGPT [0.0]
  This research delves into the construction and utilization of synthetic datasets, specifically within the telematics sphere, leveraging OpenAI's powerful language model, ChatGPT. To illustrate this data creation process, a hands-on case study is conducted, focusing on the generation of a synthetic telematics dataset.
  arXiv  Detail & Related papers  (2023-06-23T15:15:13Z)
• Forecasting Future World Events with Neural Networks [68.43460909545063]
  Autocast is a dataset containing thousands of forecasting questions and an accompanying news corpus. The news corpus is organized by date, allowing us to precisely simulate the conditions under which humans made past forecasts. We test language models on our forecasting task and find that performance is far below a human expert baseline.
  arXiv  Detail & Related papers  (2022-06-30T17:59:14Z)
• Robust Event Classification Using Imperfect Real-world PMU Data [58.26737360525643]
  We study robust event classification using imperfect real-world phasor measurement unit (PMU) data. We develop a novel machine learning framework for training robust event classifiers.
  arXiv  Detail & Related papers  (2021-10-19T17:41:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.