Reinforcement Learning for LLM-based Event Forecasting

• URL: http://arxiv.org/abs/2606.15917v1
• Date: Sun, 14 Jun 2026 16:46:11 GMT
• Title: Reinforcement Learning for LLM-based Event Forecasting
• Abstract summary: Group Relative Policy Optimization (GRPO) was recently devised sample and memory efficient reinforcement learning method.<n>We use it to finetune pretrained LLMs in the range of 1.5B to 14B parameters equipped with the ability to get current information.<n>We bring a 1.5B parameter transformer (Qwen 2.5 1.5B) to forecasting performance superior to Claude Sonnet 3.5 over the same dataset.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We use Group Relative Policy Optimization (GRPO), a recently devised sample and memory efficient reinforcement learning method, to finetune pretrained LLMs in the range of 1.5B to 14B parameters equipped with the ability to get current information through the use of a Wikipedia revisions tool, or news summaries, to forecast real events beyond the knowledge cutoff of the LLM, as well as problems made to simulate different aspects of the dynamics of that training. We use the results of these experiments to comment on the scaling capability of LLMs for forecasting, as well as classify how judgmental forecasting fits into the verifiable/unverifiable domain taxonomy, considering the impact of the inherent aleatoric uncertainty when forecasting future events (e.g. the roll of a die). As a result of the GRPO training, we manage to bring a 1.5B parameter transformer (Qwen 2.5 1.5B) to forecasting performance superior to Claude Sonnet 3.5 over the same dataset as measured by cross entropy from the market agreed probabilities. We also discuss various dead ends on the path to this result.

Related papers

• LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications [76.41731220830714]
  Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external tools, and iterative prediction.<n>We examine applications in finance, weather, health, energy, and operations, and we summarize the benchmarks and datasets used for evaluation.
  arXiv  Detail & Related papers  (2026-08-24T10:02:39Z)
• LEAF: A Living Benchmark for Event-Augmented Forecasting [78.94019460534442]
  LEAF is the first living benchmark for event-augmented forecasting tasks.<n>We evaluate state-of-the-art proprietary and open-weight LLMs.<n>We find that LLMs achieve better performance on equities they confidently identify as more predictable.
  arXiv  Detail & Related papers  (2026-05-09T03:17:59Z)
• The Forecast Critic: Leveraging Large Language Models for Poor Forecast Identification [74.64864354503204]
  We propose The Forecast Critic, a system that leverages Large Language Models (LLMs) for automated forecast monitoring.<n>We evaluate the ability of LLMs to assess time series forecast quality.<n>We present three experiments, including on both synthetic and real-world forecasting data.
  arXiv  Detail & Related papers  (2025-12-12T21:59:53Z)
• Bridging the Gap Between Bayesian Deep Learning and Ensemble Weather Forecasts [100.26854618129039]
  Weather forecasting is fundamentally challenged by the chaotic nature of the atmosphere.<n>Recent advances in Bayesian Deep Learning (BDL) offer a promising but often disconnected alternative.<n>We bridge these paradigms through a unified hybrid BDL framework for ensemble weather forecasting.
  arXiv  Detail & Related papers  (2025-11-18T07:49:52Z)
• Using LLMs to Directly Guess Conditional Expectations Can Improve Efficiency in Causal Estimation [0.3222802562733787]
  We show that predictions made by generative models trained on historical data can be used to improve the performance of these estimators.<n>We consider a case study using a small dataset of online jewelry auctions, and demonstrate that inclusion of LLM-generated guesses as predictors can improve efficiency in estimation.
  arXiv  Detail & Related papers  (2025-10-09T03:34:06Z)
• Outcome-based Reinforcement Learning to Predict the Future [1.4313866885019229]
  We show that a compact (14B) reasoning model can be trained to match or surpass the predictive accuracy of frontier models like o1.<n>The model's performance is also practically meaningful: in a Polymarket trading simulation, we estimate that its bets would have yielded a return on investment of over 10%.
  arXiv  Detail & Related papers  (2025-05-23T14:56:07Z)
• Scalable Multi-Stage Influence Function for Large Language Models via Eigenvalue-Corrected Kronecker-Factored Parameterization [31.379237532476875]
  Pre-trained large language models (LLMs) are commonly fine-tuned to adapt to downstream tasks.<n>In this paper, we propose a multi-stage influence function to attribute predictions of fine-tuned LLMs to pre-training data.
  arXiv  Detail & Related papers  (2025-05-08T07:43:44Z)
• Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs? [42.608899417822656]
  We construct a dataset using 50 1B parameter LLM variants with systematically varied pre-training configurations.<n>We introduce novel unsupervised and supervised proxy metrics derived from pre-training that successfully reduce the relative performance prediction error rate by over 50%.
  arXiv  Detail & Related papers  (2025-04-16T21:19:09Z)
• Scaling Laws for Predicting Downstream Performance in LLMs [75.28559015477137]
  This work focuses on the pre-training loss as a more computation-efficient metric for performance estimation.<n>We present FLP-M, a fundamental approach for performance prediction that addresses the practical need to integrate datasets from multiple sources during pre-training.
  arXiv  Detail & Related papers  (2024-10-11T04:57:48Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.