PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction

• URL: http://arxiv.org/abs/2601.06088v1
• Date: Wed, 31 Dec 2025 08:35:46 GMT
• Title: PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction
• Authors: Bohan Liang, Zijian Chen, Qi Jia, Kaiwei Zhang, Kaiyuan Ji, Guangtao Zhai,
• Abstract summary: We introduce PriceSeer, a benchmark specifically designed for large language models performing stock prediction tasks.<n>PriceSeer includes 110 U.S. stocks from 11 industrial sectors, with each containing 249 historical data points.<n>We evaluate six cutting-edge LLMs under different prediction horizons, demonstrating their potential in generating investment strategies.
• Score: 47.70107097572211
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stock prediction, a subject closely related to people's investment activities in fully dynamic and live environments, has been widely studied. Current large language models (LLMs) have shown remarkable potential in various domains, exhibiting expert-level performance through advanced reasoning and contextual understanding. In this paper, we introduce PriceSeer, a live, dynamic, and data-uncontaminated benchmark specifically designed for LLMs performing stock prediction tasks. Specifically, PriceSeer includes 110 U.S. stocks from 11 industrial sectors, with each containing 249 historical data points. Our benchmark implements both internal and external information expansion, where LLMs receive extra financial indicators, news, and fake news to perform stock price prediction. We evaluate six cutting-edge LLMs under different prediction horizons, demonstrating their potential in generating investment strategies after obtaining accurate price predictions for different sectors. Additionally, we provide analyses of LLMs' suboptimal performance in long-term predictions, including the vulnerability to fake news and specific industries. The code and evaluation data will be open-sourced at https://github.com/BobLiang2113/PriceSeer.

Related papers

• CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrency [60.83660377169452]
  This paper introduces CryptoBench, the first expert-curated, dynamic benchmark designed to rigorously evaluate the real-world capabilities of Large Language Model (LLM) agents.<n>Unlike general-purpose agent benchmarks for search and prediction, professional crypto analysis presents specific challenges.
  arXiv  Detail & Related papers  (2025-11-29T09:52:34Z)
• FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction [92.7392863957204]
  FutureX is the largest and most diverse live benchmark for future prediction.<n>It supports real-time daily updates and eliminates data contamination through an automated pipeline for question gathering and answer collection.<n>We evaluate 25 LLM/agent models, including those with reasoning, search capabilities, and integration of external tools.
  arXiv  Detail & Related papers  (2025-08-16T08:54:08Z)
• Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents [69.58565132975504]
  Large language models (LLMs) have demonstrated remarkable capabilities in natural language tasks.<n>We present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading.
  arXiv  Detail & Related papers  (2025-02-25T08:41:01Z)
• FinGPT: Enhancing Sentiment-Based Stock Movement Prediction with Dissemination-Aware and Context-Enriched LLMs [8.303827622646086]
  We propose a data-driven approach that enhances sentiment-based stock movement predictions by incorporating news dissemination breadth, contextual data, and explicit instructions.<n>Our approach improves prediction accuracy by 8% compared to existing methods.
  arXiv  Detail & Related papers  (2024-12-14T13:04:42Z)
• News-Driven Stock Price Forecasting in Indian Markets: A Comparative Study of Advanced Deep Learning Models [0.0]
  Recent advancements in artificial intelligence (AI) and natural language processing (NLP) have significantly enhanced stock price prediction capabilities. We leverage 30 years of historical data from national banks in India, sourced from the National Stock Exchange, to forecast stock prices. Our approach utilizes state-of-the-art deep learning models, including multivariate multi-step Long Short-Term Memory (LSTM), Facebook Prophet with LightGBM optimized through Optuna, and Seasonal Auto-Regressive Integrated Moving Average (SARIMA)
  arXiv  Detail & Related papers  (2024-10-14T15:30:06Z)
• Enhancing Few-Shot Stock Trend Prediction with Large Language Models [9.439423168290011]
  Existing methods mostly focus on predicting stock trends with supervised models trained on extensive annotated data. We propose using Large Language Models (LLMs) in a few-shot setting to overcome the scarcity of labeled data. Our method achieves 66.59% accuracy in S&P 500, 62.17% in CSI-100, and 61.17% in HK stock prediction.
  arXiv  Detail & Related papers  (2024-07-12T05:43:11Z)
• LLMFactor: Extracting Profitable Factors through Prompts for Explainable Stock Movement Prediction [5.519288891583653]
  We introduce a novel framework called LLMFactor to identify factors that influence stock movements. Unlike previous methods that relied on keyphrases or sentiment analysis, this approach focuses on extracting factors more directly related to stock market dynamics. Our framework directs the LLMs to create background knowledge through a fill-in-the-blank strategy and then discerns potential factors affecting stock prices from related news.
  arXiv  Detail & Related papers  (2024-06-16T06:20:50Z)
• Exploring Sectoral Profitability in the Indian Stock Market Using Deep Learning [0.0]
  The study builds upon existing literature on stock price prediction methods, emphasizing the shift toward machine learning and deep learning approaches. Using historical stock prices of 180 stocks across 18 sectors listed on the NSE, India, the LSTM model predicts future prices. Results demonstrate the efficacy of the LSTM model in accurately predicting stock prices and informing investment decisions.
  arXiv  Detail & Related papers  (2024-05-28T17:55:54Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [48.87381259980254]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock market reactions from news headlines without direct financial training.<n>Using post-knowledge-cutoff headlines, GPT-4 captures initial market responses, achieving approximately 90% portfolio-day hit rates for the non-tradable initial reaction.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.