SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection

• URL: http://arxiv.org/abs/2505.14420v2
• Date: Tue, 07 Oct 2025 14:03:55 GMT
• Title: SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection
• Authors: Huopu Zhang, Yanguang Liu, Miao Zhang, Zirui He, Mengnan Du,
• Abstract summary: We propose the SAE-FiRE (Sparse Autoencoder for Financial Representation Enhancement) framework to address these limitations.<n> SAE-FiRE employs Sparse Autoencoders (SAEs) to decompose dense neural representations from large language models into interpretable sparse components.<n>By systematically filtering out noise that might otherwise lead to overfitting, we enable more robust and generalizable predictions.
• Score: 29.540850801930276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting earnings surprises from financial documents, such as earnings conference calls, regulatory filings, and financial news, has become increasingly important in financial economics. However, these financial documents present significant analytical challenges, typically containing over 5,000 words with substantial redundancy and industry-specific terminology that creates obstacles for language models. In this work, we propose the SAE-FiRE (Sparse Autoencoder for Financial Representation Enhancement) framework to address these limitations by extracting key information while eliminating redundancy. SAE-FiRE employs Sparse Autoencoders (SAEs) to decompose dense neural representations from large language models into interpretable sparse components, then applies statistical feature selection methods, including ANOVA F-tests and tree-based importance scoring, to identify the top-k most discriminative dimensions for classification. By systematically filtering out noise that might otherwise lead to overfitting, we enable more robust and generalizable predictions. Experimental results across three financial datasets demonstrate that SAE-FiRE significantly outperforms baseline approaches.

Related papers

• Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection [17.04809129025246]
  FinFRE-RAG is a two-stage approach that applies importance-guided feature reduction to serialize a compact subset of numeric/categorical attributes into natural language.<n>LLMs can produce human-readable explanations and facilitate feature analysis, potentially reducing the manual workload of fraud analysts.
  arXiv  Detail & Related papers  (2025-12-15T07:09:11Z)
• FinSight: Towards Real-World Financial Deep Research [68.31086471310773]
  FinSight is a novel framework for producing high-quality, multimodal financial reports.<n>To ensure professional-grade visualization, we propose an Iterative Vision-Enhanced Mechanism.<n>A two-stage Writing Framework expands concise Chain-of-Analysis segments into coherent, citation-aware, and multimodal reports.
  arXiv  Detail & Related papers  (2025-10-19T14:05:35Z)
• Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models [0.06749750044497731]
  This paper focuses on representation bias in open-source Qwen models.<n>Using statistical tests and variance analysis, we find that firm size and valuation consistently increase model confidence.<n>When models are prompted for specific financial categories, their confidence rankings best align with fundamental data, moderately with technical signals, and least with growth indicators.
  arXiv  Detail & Related papers  (2025-10-07T09:10:13Z)
• FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering [57.18367828883773]
  FinAgentBench is a benchmark for evaluating agentic retrieval with multi-step reasoning in finance.<n>The benchmark consists of 26K expert-annotated examples on S&P-500 listed firms.<n>We evaluate a suite of state-of-the-art models and demonstrate how targeted fine-tuning can significantly improve agentic retrieval performance.
  arXiv  Detail & Related papers  (2025-08-07T22:15:22Z)
• FinAI-BERT: A Transformer-Based Model for Sentence-Level Detection of AI Disclosures in Financial Reports [6.324803752309524]
  This study introduces FinAI-BERT, a domain-adapted transformer-based language model designed to classify AI-related content at the sentence level within financial texts.<n>The model was fine-tuned on a manually curated and balanced dataset of 1,586 sentences drawn from 669 annual reports of U.S. banks.
  arXiv  Detail & Related papers  (2025-06-29T09:33:29Z)
• Truth in the Few: High-Value Data Selection for Efficient Multi-Modal Reasoning [71.3533541927459]
  We propose a novel data selection paradigm termed Activation Reasoning Potential (RAP)<n>RAP identifies cognitive samples by estimating each sample's potential to stimulate genuine multi-modal reasoning.<n>Our RAP method consistently achieves superior performance using only 9.3% of the training data, while reducing computational costs by over 43%.
  arXiv  Detail & Related papers  (2025-06-05T08:40:24Z)
• Evaluating Financial Sentiment Analysis with Annotators Instruction Assisted Prompting: Enhancing Contextual Interpretation and Stock Prediction Accuracy [1.481550828146527]
  Annotators' Instruction Assisted Prompt (AIAP) aims to standardize the understanding of sentiment across both human and machine interpretations.<n>We utilize a new dataset, WSBS, derived from the WallStreetBets subreddit to demonstrate how AIAP significantly enhances LLM performance.<n>This context-aware approach yields incremental gains in performance and also introduces an innovative sentiment-indexing method.
  arXiv  Detail & Related papers  (2025-05-09T19:44:04Z)
• FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation [65.04104723843264]
  We present FinDER, an expert-generated dataset tailored for Retrieval-Augmented Generation (RAG) in finance.<n>FinDER focuses on annotating search-relevant evidence by domain experts, offering 5,703 query-evidence-answer triplets.<n>By challenging models to retrieve relevant information from large corpora, FinDER offers a more realistic benchmark for evaluating RAG systems.
  arXiv  Detail & Related papers  (2025-04-22T11:30:13Z)
• Supervised Optimism Correction: Be Confident When LLMs Are Sure [91.7459076316849]
  We establish a novel theoretical connection between supervised fine-tuning and offline reinforcement learning.<n>We show that the widely used beam search method suffers from unacceptable over-optimism.<n>We propose Supervised Optimism Correction, which introduces a simple yet effective auxiliary loss for token-level $Q$-value estimations.
  arXiv  Detail & Related papers  (2025-04-10T07:50:03Z)
• Explainable Artificial Intelligence for identifying profitability predictors in Financial Statements [0.7067443325368975]
  We apply Machine Learning techniques to raw financial statements data taken from AIDA, a Database comprising Italian listed companies' data from 2013 to 2022.<n>We present a comparative study of different models and following the European AI regulations, we complement our analysis by applying explainability techniques to the proposed models.
  arXiv  Detail & Related papers  (2025-01-29T14:33:23Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• ECC Analyzer: Extract Trading Signal from Earnings Conference Calls using Large Language Model for Stock Performance Prediction [7.358590821647365]
  This research introduces a novel framework: textbfECC Analyzer, which utilizes large language models (LLMs) to extract richer, more predictive content from ECCs. We use the pre-trained large models to extract textual and audio features from ECCs and implement a hierarchical information extraction strategy to extract more fine-grained information. Experimental results demonstrate that our model outperforms traditional analytical benchmarks.
  arXiv  Detail & Related papers  (2024-04-29T07:11:39Z)
• Towards reducing hallucination in extracting information from financial reports using Large Language Models [1.2289361708127877]
  We show how Large Language Models (LLMs) can efficiently and rapidly extract information from earnings report transcripts. We evaluate the outcomes of various LLMs with and without using our proposed approach based on various objective metrics for evaluating Q&A systems.
  arXiv  Detail & Related papers  (2023-10-16T18:45:38Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [51.3422222472898]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock price movements using news headlines. We develop a theoretical model incorporating information capacity constraints, underreaction, limits-to-arbitrage, and LLMs.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)
• Semantic Information Marketing in The Metaverse: A Learning-Based Contract Theory Framework [68.8725783112254]
  We address the problem of designing incentive mechanisms by a virtual service provider (VSP) to hire sensing IoT devices to sell their sensing data. Due to the limited bandwidth, we propose to use semantic extraction algorithms to reduce the delivered data by the sensing IoT devices. We propose a novel iterative contract design and use a new variant of multi-agent reinforcement learning (MARL) to solve the modelled multi-dimensional contract problem.
  arXiv  Detail & Related papers  (2023-02-22T15:52:37Z)
• Multi-Domain Transformer-Based Counterfactual Augmentation for Earnings Call Analysis [20.087027853160627]
  We propose a transformer-based EC encoder to attentively quantify the task-inspired significance of critical EC content for market inference. We then develop a multi-domain counterfactual learning framework to evaluate the gradient-based variations. Experiments on the real-world financial datasets demonstrate the effectiveness of interpretable MTCA for improving the volatility evaluation ability of the state-of-the-art by 14.2% in accuracy.
  arXiv  Detail & Related papers  (2021-12-02T03:40:17Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• FinQA: A Dataset of Numerical Reasoning over Financial Data [52.7249610894623]
  We focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. We propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge.
  arXiv  Detail & Related papers  (2021-09-01T00:08:14Z)
• Towards Earnings Call and Stock Price Movement [7.196468151661785]
  We propose to model the language in transcripts using a deep learning framework. We show that the proposed model is superior to the traditional machine learning baselines.
  arXiv  Detail & Related papers  (2020-08-23T20:38:14Z)
• Foreseeing the Benefits of Incidental Supervision [83.08441990812636]
  This paper studies whether we can, in a single framework, quantify the benefits of various types of incidental signals for a given target task without going through experiments. We propose a unified PAC-Bayesian motivated informativeness measure, PABI, that characterizes the uncertainty reduction provided by incidental supervision signals.
  arXiv  Detail & Related papers  (2020-06-09T20:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.