Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators

• URL: http://arxiv.org/abs/2608.30110v1
• Date: Mon, 31 Aug 2026 00:53:28 GMT
• Title: Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators
• Abstract summary: Large language model (LLM) agents are a promising candidate for nowcasting headline macroeconomic indicators.<n>LiveMacroEval is a live, contamination-resistant benchmark in which LLM agents produce hourly nowcasts for sixteen major U.S. macroeconomic indicators.<n>Nowcast quality is assessed through a LiveMacro Score against announcement-window equity returns and a LiveBetting Score from simulated Polymarket-style trading.
• Score: 28.405337173390247
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nowcasting headline macroeconomic indicators, i.e., estimating an indicator's value for the current reference period before its official release, is critical for monetary policy and financial markets, and central banks devote dedicated teams of expert economists to producing such estimates. Large language model (LLM) agents are a promising candidate for this task, combining broad world knowledge with real-time web search and supporting queries at higher frequency than institutional nowcasts. Evaluating their nowcasting capability is, however, challenging: headline indicators such as GDP and CPI are widely reported and likely memorized during pretraining, so any evaluation on historical releases is vulnerable to data contamination. To address this, we introduce LiveMacroEval, a live, contamination-resistant benchmark in which LLM agents produce hourly nowcasts for sixteen major U.S. macroeconomic indicators over a pre-release window closing at each official release. Nowcast quality is assessed through a LiveMacro Score against announcement-window equity returns and a LiveBetting Score from simulated Polymarket-style trading, with Federal Reserve regional-bank nowcasts, the Bloomberg ECOS professional consensus, and an auto-ARIMA baseline as comparators. Over six months with four state-of-the-art LLM agents configured with web search, aggregate nowcast accuracy is broadly comparable to the institutional and professional benchmarks, with performance varying widely across individual indicators. This highlights LLM agents' potential as real-time estimators of macroeconomic conditions.

Related papers

• FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Construction [57.909515451254855]
  FinDeepIndicator is the first benchmark dedicated to evaluating Deep Research (DR) agents in end-to-end financial indicator construction.<n>It evaluates DR agents across four stages in indicator construction: formula specification, data collection, indicator calculation, and answer generation.<n>It contains 3,350 curated question-answer pairs derived from both U.S. and Chinese markets, 10 years of historical financial data, and 800 listed companies.
  arXiv  Detail & Related papers  (2026-08-01T16:54:41Z)
• Nexus : An Agentic Framework for Time Series Forecasting [72.73790673303154]
  Time series forecasting requires reasoning with unstructured contextual data such as news or events.<n>We introduce Nexus, a multi-agent forecasting framework that decomposes prediction into specialized stages.<n>We show that current-generation LLMs possess substantially stronger intrinsic forecasting ability than previously recognized.
  arXiv  Detail & Related papers  (2026-05-14T05:12:13Z)
• CN-Buzz2Portfolio: A Chinese-Market Dataset and Benchmark for LLM-Based Macro and Sector Asset Allocation from Daily Trending Financial News [21.498084305999992]
  Large Language Models (LLMs) are rapidly transitioning from static Natural Language Processing (NLP) tasks to acting as dynamic decision-making agents.<n>Direct live trading is irreproducible and prone to outcome bias by confounding luck with skill, whereas existing static benchmarks are often confined to entity-level stock picking and ignore broader market attention.<n>This work provides new insights into the alignment between general reasoning and financial decision-making, and all data, codes, and experiments are released to promote sustainable financial agent research.
  arXiv  Detail & Related papers  (2026-03-18T02:31:28Z)
• PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction [47.70107097572211]
  We introduce PriceSeer, a benchmark specifically designed for large language models performing stock prediction tasks.<n>PriceSeer includes 110 U.S. stocks from 11 industrial sectors, with each containing 249 historical data points.<n>We evaluate six cutting-edge LLMs under different prediction horizons, demonstrating their potential in generating investment strategies.
  arXiv  Detail & Related papers  (2025-12-31T08:35:46Z)
• CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrency [60.83660377169452]
  This paper introduces CryptoBench, the first expert-curated, dynamic benchmark designed to rigorously evaluate the real-world capabilities of Large Language Model (LLM) agents.<n>Unlike general-purpose agent benchmarks for search and prediction, professional crypto analysis presents specific challenges.
  arXiv  Detail & Related papers  (2025-11-29T09:52:34Z)
• How AI Forecasts AI Jobs: Benchmarking LLM Predictions of Labor Market Changes [5.848712585343904]
  This paper introduces a benchmark for evaluating how well large language models (LLMs) can anticipate changes in job demand.<n>Our benchmark combines two datasets: a high-frequency index of sector-level job postings in the United States, and a global dataset of projected occupational changes due to AI adoption.<n>Results show that structured task prompts consistently improve forecast stability, while persona prompts offer advantages on short-term trends.
  arXiv  Detail & Related papers  (2025-10-27T14:08:27Z)
• StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets? [44.10622904101254]
  Large language models (LLMs) have recently demonstrated strong capabilities as autonomous agents.<n>We introduce StockBench, a benchmark designed to evaluate LLM agents in realistic, multi-month stock trading environments.<n>Our evaluation shows that while most LLM agents struggle to outperform the simple buy-and-hold baseline, several models demonstrate the potential to deliver higher returns and manage risk more effectively.
  arXiv  Detail & Related papers  (2025-10-02T16:54:57Z)
• FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction [92.7392863957204]
  FutureX is the largest and most diverse live benchmark for future prediction.<n>It supports real-time daily updates and eliminates data contamination through an automated pipeline for question gathering and answer collection.<n>We evaluate 25 LLM/agent models, including those with reasoning, search capabilities, and integration of external tools.
  arXiv  Detail & Related papers  (2025-08-16T08:54:08Z)
• Can We Reliably Predict the Fed's Next Move? A Multi-Modal Approach to U.S. Monetary Policy Forecasting [2.6396287656676733]
  This study examines whether predictive accuracy can be enhanced by integrating structured data with unstructured textual signals from Federal Reserve communications.<n>Our results show that hybrid models consistently outperform unimodal baselines.<n>For monetary policy forecasting, simpler hybrid models can offer both accuracy and interpretability, delivering actionable insights for researchers and decision-makers.
  arXiv  Detail & Related papers  (2025-06-28T05:54:58Z)
• Macroeconomic Forecasting with Large Language Models [2.5234156040689233]
  This paper presents a comparative analysis evaluating the accuracy of Large Language Models (LLMs) against traditional macro time series forecasting approaches.<n>Our findings provide valuable insights into the strengths and limitations of LLMs in forecasting macroeconomic time series, shedding light on their applicability in real-world scenarios.
  arXiv  Detail & Related papers  (2024-07-01T01:25:26Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [48.87381259980254]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock market reactions from news headlines without direct financial training.<n>Using post-knowledge-cutoff headlines, GPT-4 captures initial market responses, achieving approximately 90% portfolio-day hit rates for the non-tradable initial reaction.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.