FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewards

• URL: http://arxiv.org/abs/2604.26733v2
• Date: Thu, 07 May 2026 14:24:24 GMT
• Title: FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewards
• Abstract summary: We present FutureWorld, a live agentic reinforcement learning environment that closes the training loop between prediction, outcome realization, and parameter updates.<n>Across three open-source agents, successive FutureWorld training rounds lead to consistent improvements in prediction accuracy, probabilistic scoring, and calibration.
• Score: 20.541743597851177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Live future prediction refers to the task of making predictions about real-world events before they unfold. This task is increasingly studied using large language model-based agent systems, and it is important for building agents that can continually learn from the real world. It can provide a large number of prediction questions grounded in diverse real-world events, while preventing answer leakage. To leverage the advantages of future prediction, we present FutureWorld, a live agentic reinforcement learning environment that closes the training loop between prediction, outcome realization, and parameter updates. Specifically, we modify and extend verl-tool, resulting in a new framework that we call verl-tool-future. Unlike standard RL training frameworks that rely on immediate rewards, verl-tool-future stores prediction-time rollouts, backfills rewards after real-world outcomes become available, and then replays the completed trajectories for policy update. Across three open-source agents, successive FutureWorld training rounds lead to consistent improvements in prediction accuracy, probabilistic scoring, and calibration, demonstrating that delayed real-world outcome feedback can serve as an effective RL signal for predictive agents.

Related papers

• Self-Evolving World Models for LLM Agent Planning [72.99782619992361]
  We introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen.<n>WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate.
  arXiv  Detail & Related papers  (2026-06-29T17:58:43Z)
• FutureSim: Replaying World Events to Evaluate Adaptive Agents [69.52411849743358]
  We build FutureSim, where agents forecast world events beyond their knowledge cutoff while interacting with a chronological replay of the world.<n>FutureSim reveals a clear separation in their capabilities, with the best agent's accuracy being 25%, and many having worse Brier skill score than making no prediction at all.<n>Overall, we hope our benchmark design paves the way to measure AI progress on open-ended adaptation spanning long time-horizons in the real world.
  arXiv  Detail & Related papers  (2026-05-14T17:59:28Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• Future-as-Label: Scalable Supervision from Real-World Outcomes [0.0]
  Time creates free supervision: forecasts about real-world events resolve to verifiable outcomes.<n>We extend reinforcement learning with verifiable rewards to real-world prediction over time.<n>We train language models to make probabilistic forecasts from causally masked information.
  arXiv  Detail & Related papers  (2026-01-09T22:15:12Z)
• Current Agents Fail to Leverage World Model as Tool for Foresight [61.82522354207919]
  Generative world models offer a promising remedy: agents could use them to foresee outcomes before acting.<n>This paper empirically examines whether current agents can leverage such world models as tools to enhance their cognition.
  arXiv  Detail & Related papers  (2026-01-07T13:15:23Z)
• Scaling Open-Ended Reasoning to Predict the Future [56.672065928345525]
  We train language models to make predictions on open-ended forecasting questions.<n>To scale up training data, we synthesize novel forecasting questions from global events reported in daily news.<n>We find calibration improvements from forecasting training generalize across popular benchmarks.
  arXiv  Detail & Related papers  (2025-12-31T18:59:51Z)
• Predicting the Future by Retrieving the Past [23.181638022094038]
  We propose Predicting the Future by Retrieving the Past (PFRP), a novel approach that explicitly integrates global historical data to enhance forecasting accuracy.<n>PFRP produces more accurate and interpretable forecasts by adaptively combining global predictions with the outputs of any local prediction model.
  arXiv  Detail & Related papers  (2025-11-08T05:24:45Z)
• Outcome-based Reinforcement Learning to Predict the Future [1.4313866885019229]
  We show that a compact (14B) reasoning model can be trained to match or surpass the predictive accuracy of frontier models like o1.<n>The model's performance is also practically meaningful: in a Polymarket trading simulation, we estimate that its bets would have yielded a return on investment of over 10%.
  arXiv  Detail & Related papers  (2025-05-23T14:56:07Z)
• Wisdom of the Crowds in Forecasting: Forecast Summarization for Supporting Future Event Prediction [17.021220773165016]
  Future Event Prediction (FEP) is an essential activity whose demand and application range across multiple domains.<n>One forecasting way is to gather and aggregate collective opinions on the future to make predictions as cumulative perspectives carry the potential to help estimating the likelihood of upcoming events.<n>In this work, we organize the existing research and frameworks that aim to support future event prediction based on crowd wisdom through aggregating individual forecasts.
  arXiv  Detail & Related papers  (2025-02-12T08:35:10Z)
• Predictive World Models from Real-World Partial Observations [66.80340484148931]
  We present a framework for learning a probabilistic predictive world model for real-world road environments. While prior methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only.
  arXiv  Detail & Related papers  (2023-01-12T02:07:26Z)
• Palm up: Playing in the Latent Manifold for Unsupervised Pretraining [31.92145741769497]
  We propose an algorithm that exhibits an exploratory behavior whilst it utilizes large diverse datasets. Our key idea is to leverage deep generative models that are pretrained on static datasets and introduce a dynamic model in the latent space. We then employ an unsupervised reinforcement learning algorithm to explore in this environment and perform unsupervised representation learning on the collected data.
  arXiv  Detail & Related papers  (2022-10-19T22:26:12Z)
• What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience [63.75363908696257]
  computational reinforcement learning seeks to construct an agent's perception of the world through predictions of future sensations. An open challenge in this line of work is determining from the infinitely many predictions that the agent could possibly make which predictions might best support decision-making. We introduce a meta-gradient descent process by which an agent learns what predictions to make, 2) the estimates for its chosen predictions, and 3) how to use those estimates to generate policies that maximize future reward.
  arXiv  Detail & Related papers  (2022-06-13T21:31:06Z)
• A-ACT: Action Anticipation through Cycle Transformations [89.83027919085289]
  We take a step back to analyze how the human capability to anticipate the future can be transferred to machine learning algorithms. A recent study on human psychology explains that, in anticipating an occurrence, the human brain counts on both systems. In this work, we study the impact of each system for the task of action anticipation and introduce a paradigm to integrate them in a learning framework.
  arXiv  Detail & Related papers  (2022-04-02T21:50:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.