Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements

• URL: http://arxiv.org/abs/2609.03654v1
• Date: Thu, 03 Sep 2026 10:52:56 GMT
• Title: Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements
• Abstract summary: We introduce FinRAG-QA, a novel benchmark dataset for financial question answering.<n>FinRAG-QA targets cross-institutional retrieval over documents averaging 198k words.<n>Contextual chunk enrichment combined with a retrieval-optimised embedding model raises NDCG@10 from 0.322 to 0.710.
• Score: 6.625637186756314
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The comparative analysis of banks' financial statements poses significant challenges for automated question answering systems due to their complexity, substantial length, technical language, and inhomogeneity of both textual and numerical content across different jurisdictions and institutions. We introduce FinRAG-QA, a novel benchmark dataset for financial question answering, which comprises 999 practitioner-curated questions on 10 standardised indicators, grounded in 209 annual and Pillar 3 reports from 24 major European and U.S. banks spanning 2019-2023. Unlike prior financial QA benchmarks, which centre on U.S. filings and single-institution analysis, FinRAG-QA targets cross-institutional retrieval over documents averaging 198k words, longer than any existing financial QA resource. On this benchmark we evaluate a multi-stage RAG pipeline and isolate the contribution of each component. Contextual chunk enrichment combined with a retrieval-optimised embedding model raises NDCG@10 from 0.322 to 0.710; conditional on the ground truth being retrieved, a reasoning-optimised generator raises answer accuracy from 44.6% to 79.0% (+34.4 percentage points), at roughly 20x the generation latency. We further show that cross-encoder reranking degrades retrieval when the first-stage ranking is already strong, and that a single top-ranked chunk outperforms larger contexts at generation time. Experiments were run in late 2024-early 2025 with the models available at that time.

Related papers

• IGT @ FinMMEval 2026 Task 2: Question-Type Prompting with Targeted Extraction for Multilingual Financial QA [0.0]
  PolyFiQA Task 2 was a multilingual financial question answering task over English SEC filings and multilingual news articles.<n>Our central observation is that the 344 development questions divide into two families requiring fundamentally different approaches.<n>A dataset analysis reveals that 17-18 of 19 ground-truth reference answers per synthesis type share an exact evidence label prefix.<n>The final system achieves development ROUGE-1 approximately 0.395, a 60% relative improvement over a generic RAG baseline.
  arXiv  Detail & Related papers  (2026-09-08T02:23:31Z)
• FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Construction [57.909515451254855]
  FinDeepIndicator is the first benchmark dedicated to evaluating Deep Research (DR) agents in end-to-end financial indicator construction.<n>It evaluates DR agents across four stages in indicator construction: formula specification, data collection, indicator calculation, and answer generation.<n>It contains 3,350 curated question-answer pairs derived from both U.S. and Chinese markets, 10 years of historical financial data, and 800 listed companies.
  arXiv  Detail & Related papers  (2026-08-01T16:54:41Z)
• Hierarchical Reranking for Scalable Financial RAG System [4.690968174645844]
  We propose Hierarchical Reranker, a RAG framework designed to improve retrieval performance and generative reliability across large-scale financial datasets.<n>The proposed system achieved an NDCG@20 score of 0.7918 and demonstrated superior factual consistency.<n>This work presents a deployable, domain-optimized RAG pipeline that enhances both the accuracy and scalability of financial reasoning.
  arXiv  Detail & Related papers  (2026-07-29T23:25:45Z)
• The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems [54.12165004393043]
  FinMMEval 2026 offers three interconnected tasks that span financial understanding, reasoning, and decision-making.<n>The lab aims to promote the development of robust, transparent, and globally inclusive financial AI systems.
  arXiv  Detail & Related papers  (2026-02-11T14:14:06Z)
• AfriEconQA: A Benchmark Dataset for African Economic Analysis based on World Bank Reports [0.0]
  AfriEconQA is a specialized benchmark dataset for African economic analysis grounded in a comprehensive corpus of World Bank reports.<n>The dataset consists of 8,937 curated QA instances, rigorously filtered from a pool of 10018 synthetic questions.<n>AfriEconQA is the first benchmark focused specifically on African economic analysis, providing a unique challenge for Information Retrieval (IR) systems.
  arXiv  Detail & Related papers  (2026-01-06T00:02:11Z)
• FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [57.43420753842626]
  FinLFQA is a benchmark designed to evaluate the ability of Large Language Models to generate long-form answers to complex financial questions.<n>We provide an automatic evaluation framework covering both answer quality and attribution quality.
  arXiv  Detail & Related papers  (2025-10-07T20:06:15Z)
• FinReflectKG - MultiHop: Financial QA Benchmark for Reasoning with Knowledge Graph Evidence [0.0]
  We present FinReflectKG - MultiHop, a benchmark built on FinReflectKG, a temporally indexed financial Knowledge Graph.<n>We generate financial analyst style questions with exact supporting evidence from the KG.<n>Across both reasoning and non-reasoning models, KG-guided precise retrieval yields substantial gains.
  arXiv  Detail & Related papers  (2025-10-03T11:19:31Z)
• FinAI-BERT: A Transformer-Based Model for Sentence-Level Detection of AI Disclosures in Financial Reports [6.324803752309524]
  This study introduces FinAI-BERT, a domain-adapted transformer-based language model designed to classify AI-related content at the sentence level within financial texts.<n>The model was fine-tuned on a manually curated and balanced dataset of 1,586 sentences drawn from 669 annual reports of U.S. banks.
  arXiv  Detail & Related papers  (2025-06-29T09:33:29Z)
• FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation [65.04104723843264]
  We present FinDER, an expert-generated dataset tailored for Retrieval-Augmented Generation (RAG) in finance.<n>FinDER focuses on annotating search-relevant evidence by domain experts, offering 5,703 query-evidence-answer triplets.<n>By challenging models to retrieve relevant information from large corpora, FinDER offers a more realistic benchmark for evaluating RAG systems.
  arXiv  Detail & Related papers  (2025-04-22T11:30:13Z)
• FinTextQA: A Dataset for Long-form Financial Question Answering [10.1084081290893]
  FinTextQA is a novel dataset for long-form question answering (LFQA) in finance. The most effective system configuration on our dataset involved setting the embedder, retriever, reranker, and generator as Ada2, Automated Merged Retrieval, Bge-Reranker-Base, and Baichuan2-7B.
  arXiv  Detail & Related papers  (2024-05-16T10:53:31Z)
• FinQA: A Dataset of Numerical Reasoning over Financial Data [52.7249610894623]
  We focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. We propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge.
  arXiv  Detail & Related papers  (2021-09-01T00:08:14Z)
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [71.76018597965378]
  We build a new large-scale Question Answering dataset containing both Tabular And Textual data, named TAT-QA. We propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text.
  arXiv  Detail & Related papers  (2021-05-17T06:12:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.