Prompting for Policy: Forecasting Macroeconomic Scenarios with Synthetic LLM Personas

• URL: http://arxiv.org/abs/2511.02458v1
• Date: Tue, 04 Nov 2025 10:38:10 GMT
• Title: Prompting for Policy: Forecasting Macroeconomic Scenarios with Synthetic LLM Personas
• Abstract summary: We evaluate whether persona-based prompting improves Large Language Model (LLM) performance on macroeconomic forecasting tasks.<n>Using 2,368 economics-related personas from the PersonaHub corpus, we prompt GPT-4o to replicate the ECB Survey of Professional Forecasters.<n>We compare the persona-prompted forecasts against the human experts panel, across four target variables (HICP, core HICP, GDP growth, unemployment) and four forecast horizons.
• Score: 0.17188280334580197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We evaluate whether persona-based prompting improves Large Language Model (LLM) performance on macroeconomic forecasting tasks. Using 2,368 economics-related personas from the PersonaHub corpus, we prompt GPT-4o to replicate the ECB Survey of Professional Forecasters across 50 quarterly rounds (2013-2025). We compare the persona-prompted forecasts against the human experts panel, across four target variables (HICP, core HICP, GDP growth, unemployment) and four forecast horizons. We also compare the results against 100 baseline forecasts without persona descriptions to isolate its effect. We report two main findings. Firstly, GPT-4o and human forecasters achieve remarkably similar accuracy levels, with differences that are statistically significant yet practically modest. Our out-of-sample evaluation on 2024-2025 data demonstrates that GPT-4o can maintain competitive forecasting performance on unseen events, though with notable differences compared to the in-sample period. Secondly, our ablation experiment reveals no measurable forecasting advantage from persona descriptions, suggesting these prompt components can be omitted to reduce computational costs without sacrificing accuracy. Our results provide evidence that GPT-4o can achieve competitive forecasting accuracy even on out-of-sample macroeconomic events, if provided with relevant context data, while revealing that diverse prompts produce remarkably homogeneous forecasts compared to human panels.

Related papers

• Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking [51.56484100374058]
  We evaluate whether a fully automatic, purely feedback-driven ESN can serve as a competitive alternative to widely used statistical forecasting methods.<n>Forecast accuracy is measured using MASE and sMAPE and benchmarked against simple benchmarks like drift and seasonal naive and statistical models.
  arXiv  Detail & Related papers  (2026-02-03T16:01:22Z)
• Methodological Insights into Structural Causal Modelling and Uncertainty-Aware Forecasting for Economic Indicators [4.9930207509018425]
  We focus on four key U.S. macroeconomic indicators -- GDP, economic growth, inflation, and unemployment.<n>Our results reveal a robust unidirectional causal link from economic growth to GDP.<n>We use a large language model trained for time series to perform zero-shot predictions on unemployment.
  arXiv  Detail & Related papers  (2025-09-08T04:52:12Z)
• Consistency Checks for Language Model Forecasters [54.62507816753479]
  We measure the performance of forecasters in terms of the consistency of their predictions on different logically-related questions.<n>We build an automated evaluation system that generates a set of base questions, instantiates consistency checks from these questions, elicits predictions of the forecaster, and measures the consistency of the predictions.
  arXiv  Detail & Related papers  (2024-12-24T16:51:35Z)
• Hybrid Forecasting of Geopolitical Events [71.73737011120103]
  SAGE is a hybrid forecasting system that combines human and machine generated forecasts.<n>The system aggregates human and machine forecasts weighting both for propinquity and based on assessed skill.<n>We show that skilled forecasters who had access to machine-generated forecasts outperformed those who only viewed historical data.
  arXiv  Detail & Related papers  (2024-12-14T22:09:45Z)
• Performative Prediction on Games and Mechanism Design [69.7933059664256]
  We study a collective risk dilemma where agents decide whether to trust predictions based on past accuracy.<n>As predictions shape collective outcomes, social welfare arises naturally as a metric of concern.<n>We show how to achieve better trade-offs and use them for mechanism design.
  arXiv  Detail & Related papers  (2024-08-09T16:03:44Z)
• Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy [1.999925939110439]
  We use an ensemble approach consisting of a crowd of twelve large language models (LLMs) We compare the aggregated LLM predictions on 31 binary questions to that of a crowd of human forecasters from a three-month forecasting tournament. We find that both models' forecasting accuracy benefits from exposure to the median human prediction as information.
  arXiv  Detail & Related papers  (2024-02-29T17:27:59Z)
• Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament [2.900810893770134]
  We enroll OpenAI's state-of-the-art large language model, GPT-4, in a three-month forecasting tournament hosted on the Metaculus platform. We show that GPT-4's probabilistic forecasts are significantly less accurate than the median human-crowd forecasts. A potential explanation for this underperformance is that in real-world forecasting tournaments, the true answers are genuinely unknown at the time of prediction.
  arXiv  Detail & Related papers  (2023-10-17T17:58:17Z)
• Towards More Fine-grained and Reliable NLP Performance Prediction [85.78131503006193]
  We make two contributions to improving performance prediction for NLP tasks. First, we examine performance predictors for holistic measures of accuracy like F1 or BLEU. Second, we propose methods to understand the reliability of a performance prediction model from two angles: confidence intervals and calibration.
  arXiv  Detail & Related papers  (2021-02-10T15:23:20Z)
• A generative adversarial network approach to (ensemble) weather prediction [91.3755431537592]
  We use a conditional deep convolutional generative adversarial network to predict the geopotential height of the 500 hPa pressure level, the two-meter temperature and the total precipitation for the next 24 hours over Europe. The proposed models are trained on 4 years of ERA5 reanalysis data from 2015-2018 with the goal to predict the associated meteorological fields in 2019.
  arXiv  Detail & Related papers  (2020-06-13T20:53:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.