Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls

• URL: http://arxiv.org/abs/2605.03147v1
• Date: Mon, 04 May 2026 20:40:05 GMT
• Title: Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls
• Abstract summary: We assess the information captured by models trained on SEC filings and in-context learning methods.<n>We propose a system utilizing LLMs to perform open-ended extraction from unstructured call transcripts.
• Score: 43.85438181764358
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Earnings calls are a key source of financial information about public companies. However, extracting information from these calls is difficult. Unlike the templatic filings required by the U.S. Securities and Exchange Commission (SEC) to report a company's financial situation, earnings conference calls have no built-in labels, are unstructured, and feature conversational language. We explore this challenging domain by assessing the information captured by models trained on SEC filings and in-context learning methods. To establish a baseline, we first evaluate the generalization capabilities of SEC-trained models across established SEC datasets. To support our investigation, we introduce three novel benchmarks: (1) SEC Filings Benchmark (SECB), (2) Earnings Calls Benchmark (ECB), and ECB-A, a subset with 2,460 expert annotation groups to support our qualitative analysis. We find that encoder-based models struggle with the domain shift. Finally, we propose a system utilizing LLMs to perform open-ended extraction from unstructured call transcripts, verified by human evaluation (79.7% precision), providing a baseline for this valuable domain through the consistent tracking of emergent KPIs.

Related papers

• LLM-Based Examination of Eligibility Criteria from Securities Prospectuses at the German Central Bank [1.8723525304649937]
  We present the first case study applying Large Language Models (LLMs) to the eligibility examination process.<n>Our approach decomposes the task into extraction, normalization, and interpretation, allowing for greater flexibility in handling noisy text.<n>Our results demonstrate that LLM-based systems achieve high precision (up to 91%) in document-level eligibility.
  arXiv  Detail & Related papers  (2026-06-25T17:29:58Z)
• The Stanford EDGAR Filings Dataset: Reconstructing U.S. Corporate and Financial Disclosures into Layout-Faithful and Token-Efficient Pretraining Data [1.6404022072626985]
  Stanford EDGAR Filings dataset is a reconstruction of SEC filings into layout-faithful MultiMarkdown.<n>SEFD makes audited financial statements, risk disclosures, ownership reports, accounting notes, and market-moving event filings usable as long-context pretraining data.<n>We release SEFD-v1, a 152B-token initial public snapshot, and provide corpus-level analyses of a larger 18.5M-filing archive estimated at 550B tokens.
  arXiv  Detail & Related papers  (2026-06-16T17:22:34Z)
• Automated Benchmark Auditing for AI Agents and Large Language Models [46.03841647776303]
  Auto Benchmark Audit (ABA) is an agentic framework that systematically audits individual benchmark tasks.<n>We run ABA on a collection of frontier LLM benchmarks and previous NeurIPS publications, totaling 168 benchmarks across nine domains.<n>ABA identifies critical issues including ambiguous task design, execution environment conflicts, and incorrect ground truths in over 25.7% of the evaluated tasks.
  arXiv  Detail & Related papers  (2026-05-25T17:44:21Z)
• Enhancing Business Analytics through Hybrid Summarization of Financial Reports [0.152292571922932]
  Financial reports and earnings communications contain large volumes of structured and semi structured information.<n>We present a hybrid summarization framework that combines extractive and abstractive techniques to produce concise and factually reliable summaries.<n>These findings support the development of practical summarization systems for distilling lengthy financial texts into usable business insights.
  arXiv  Detail & Related papers  (2025-12-28T16:25:12Z)
• Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts [45.13099538394587]
  We introduce a sentence-level corpus for stance detection focused on three core financial metrics: debt, earnings per share (EPS), and sales.<n>The sentences were extracted from Form 10-K annual reports and ECTs, and labeled for stance using the advanced ChatGPT-o3-pro model.<n>Using this corpus, we conduct a systematic evaluation of modern large language models (LLMs) using zero-shot, few-shot, and Chain-of-Thought (CoT) prompting strategies.
  arXiv  Detail & Related papers  (2025-10-27T16:03:20Z)
• FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering [57.18367828883773]
  FinAgentBench is a benchmark for evaluating agentic retrieval with multi-step reasoning in finance.<n>The benchmark consists of 26K expert-annotated examples on S&P-500 listed firms.<n>We evaluate a suite of state-of-the-art models and demonstrate how targeted fine-tuning can significantly improve agentic retrieval performance.
  arXiv  Detail & Related papers  (2025-08-07T22:15:22Z)
• SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection [29.540850801930276]
  We propose the SAE-FiRE (Sparse Autoencoder for Financial Representation Enhancement) framework to address these limitations.<n> SAE-FiRE employs Sparse Autoencoders (SAEs) to decompose dense neural representations from large language models into interpretable sparse components.<n>By systematically filtering out noise that might otherwise lead to overfitting, we enable more robust and generalizable predictions.
  arXiv  Detail & Related papers  (2025-05-20T14:31:23Z)
• FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecasting [58.70072722290475]
  Financial time series (FinTS) record the behavior of human-brain-augmented decision-making.<n>FinTSB is a comprehensive and practical benchmark for financial time series forecasting.
  arXiv  Detail & Related papers  (2025-02-26T05:19:16Z)
• AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework [48.3060010653088]
  We release AlphaFin datasets, combining traditional research datasets, real-time financial data, and handwritten chain-of-thought (CoT) data. We then use AlphaFin datasets to benchmark a state-of-the-art method, called Stock-Chain, for effectively tackling the financial analysis task.
  arXiv  Detail & Related papers  (2024-03-19T09:45:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.