Context-Aware Language Models for Forecasting Market Impact from Sequences of Financial News

• URL: http://arxiv.org/abs/2509.12519v1
• Date: Mon, 15 Sep 2025 23:51:13 GMT
• Title: Context-Aware Language Models for Forecasting Market Impact from Sequences of Financial News
• Authors: Ross Koval, Nicholas Andrews, Xifeng Yan,
• Abstract summary: We explore the value of historical context in the ability of large language models to understand the market impact of financial news.<n>We propose an efficient and effective contextualization method that uses a large LM to process the main article, while a small LM encodes the historical context into concise summary embeddings.
• Score: 18.185361179633553
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Financial news plays a critical role in the information diffusion process in financial markets and is a known driver of stock prices. However, the information in each news article is not necessarily self-contained, often requiring a broader understanding of the historical news coverage for accurate interpretation. Further, identifying and incorporating the most relevant contextual information presents significant challenges. In this work, we explore the value of historical context in the ability of large language models to understand the market impact of financial news. We find that historical context provides a consistent and significant improvement in performance across methods and time horizons. To this end, we propose an efficient and effective contextualization method that uses a large LM to process the main article, while a small LM encodes the historical context into concise summary embeddings that are then aligned with the large model's representation space. We explore the behavior of the model through multiple qualitative and quantitative interpretability tests and reveal insights into the value of contextualization. Finally, we demonstrate that the value of historical context in model predictions has real-world applications, translating to substantial improvements in simulated investment performance.

Related papers

• Multimodal Language Models with Modality-Specific Experts for Financial Forecasting from Interleaved Sequences of Text and Time Series [18.185361179633553]
  Text and time series data offer complementary views of financial markets.<n>We propose a unified neural architecture that models these interleaved sequences using modality-specific experts.<n>We demonstrate the effectiveness of our approach on a large-scale financial forecasting task.
  arXiv  Detail & Related papers  (2025-09-23T22:40:31Z)
• Quantifying Qualitative Insights: Leveraging LLMs to Market Predict [0.0]
  This study addresses challenges by leveraging daily reports from securities firms to create high-quality contextual information. The reports are segmented into text-based key factors and combined with numerical data, such as price information, to form context sets. A crafted prompt is designed to assign scores to the key factors, converting qualitative insights into quantitative results.
  arXiv  Detail & Related papers  (2024-11-13T07:45:40Z)
• Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
  "Context is Key" (CiK) is a forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context.<n>We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters.<n>We propose a simple yet effective LLM prompting method that outperforms all other tested methods on our benchmark.
  arXiv  Detail & Related papers  (2024-10-24T17:56:08Z)
• Context Matters: Leveraging Contextual Features for Time Series Forecasting [2.9687381456164004]
  We introduce ContextFormer, a novel plug-and-play method to surgically integrate multimodal contextual information into existing forecasting models.<n> ContextFormer effectively distills forecast-specific information from rich multimodal contexts, including categorical, continuous, time-varying, and even textual information.<n>It outperforms SOTA forecasting models by up to 30% on a range of real-world datasets spanning energy, traffic, environmental, and financial domains.
  arXiv  Detail & Related papers  (2024-10-16T15:36:13Z)
• NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities [51.07379913779232]
  NeedleBench is a framework for assessing retrieval and reasoning performance in long-context tasks.<n>It embeds key data points at varying depths to rigorously test model capabilities.<n>Our experiments reveal that reasoning models like Deep-R1 and OpenAI's o3 struggle with continuous retrieval and reasoning in information-dense scenarios.
  arXiv  Detail & Related papers  (2024-07-16T17:59:06Z)
• Detection of Temporality at Discourse Level on Financial News by Combining Natural Language Processing and Machine Learning [8.504685056067144]
  Finance-related news such as Bloomberg News, CNN Business and Forbes are valuable sources of real data for market screening systems. We propose a novel system to detect the temporality of finance-related news at discourse level. We have tested our system on a labelled dataset of finance-related news annotated by researchers with knowledge in the field.
  arXiv  Detail & Related papers  (2024-03-30T16:40:10Z)
• Enhancing Argument Structure Extraction with Efficient Leverage of Contextual Information [79.06082391992545]
  We propose an Efficient Context-aware model (ECASE) that fully exploits contextual information. We introduce a sequence-attention module and distance-weighted similarity loss to aggregate contextual information and argumentative information. Our experiments on five datasets from various domains demonstrate that our model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-08T08:47:10Z)
• Lost in the Middle: How Language Models Use Long Contexts [88.78803442320246]
  We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts. We find that performance can degrade significantly when changing the position of relevant information. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
  arXiv  Detail & Related papers  (2023-07-06T17:54:11Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [51.3422222472898]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock price movements using news headlines. We develop a theoretical model incorporating information capacity constraints, underreaction, limits-to-arbitrage, and LLMs.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Interpretable Fake News Detection with Topic and Deep Variational Models [2.15242029196761]
  We focus on fake news detection using interpretable features and methods. We have developed a deep probabilistic model that integrates a dense representation of textual news. Our model achieves comparable performance to state-of-the-art competing models.
  arXiv  Detail & Related papers  (2022-09-04T05:31:00Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.