BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents

• URL: http://arxiv.org/abs/2606.03829v1
• Date: Tue, 02 Jun 2026 16:12:34 GMT
• Title: BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents
• Abstract summary: BigFinanceBench is a 928-item benchmark of open-ended rubric financial-research tasks.<n>It decomposes the derivation into independently checkable steps.<n>It supports partial-credit evaluation and localization of failures across the analyst workflow.
• Score: 5.690389797295356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and accounting definition were used, which assumptions were made, and how the calculation was performed. Existing finance benchmarks largely evaluate isolated subskills or final answers, leaving the auditable derivation itself under-measured. We introduce BigFinanceBench, a 928-item expert-authored benchmark of open-ended financial-research tasks in which each item pairs a ground-truth reference answer with a point-weighted rubric that decomposes the derivation into independently checkable steps. BigFinanceBench is workflow-grounded in that it evaluates the full derivation rather than only the final output. Across 36,241 rubric points, the benchmark supports partial-credit evaluation and localization of failures across the analyst workflow. Evaluating ten current frontier and open-weight agents, we find substantial headroom: the best system reaches only 58.8% rubric score, final-answer accuracy is a useful but lossy proxy for derivation quality, and model capability varies non-uniformly across financial workflows.

Related papers

• FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use? [52.86361055342659]
  FinCUABuildBench is a benchmark for evaluating financial CUA task construction.<n>FinCUABuildAgent is a multi-agent system for automatically constructing dynamic financial CUA evaluation tasks.
  arXiv  Detail & Related papers  (2026-09-07T15:13:23Z)
• FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review [25.300288794507377]
  We introduce FinRiskAtlas, a Chinese-language benchmark that evaluates financial LLMs along two complementary dimensions.<n>We extend this framework through offline replay of 680 pre-action states from 104 de-identified professional trajectories.<n>FinRisk-Ask shows that entering the Ask branch more frequently does not necessarily improve request targeting or end-to-end evidence acquisition.
  arXiv  Detail & Related papers  (2026-08-26T03:13:15Z)
• Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning [0.13999481573773073]
  Hedge-Bench 1.0 is a benchmark of 102 actual, on-the-job tasks grounded in the explicit reasoning traces of professional hedge fund analysts.<n>We publish the dataset and evaluation harness at Frontier models.com/Trata-Inc/trata-hedge-bench.
  arXiv  Detail & Related papers  (2026-06-02T17:11:56Z)
• WorkstreamBench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance [4.787072076364137]
  frontier AI labs have developed agents that can construct entire spreadsheets from scratch.<n>This is especially relevant in finance, where core such as financial modeling, forecasting, and scenario analysis are commonly conducted through spreadsheets.<n>Existing spreadsheet benchmarks do not measure this advanced capability, focusing instead on question-answering or single-formula edits.
  arXiv  Detail & Related papers  (2026-05-21T16:06:34Z)
• Herculean: An Agentic Benchmark for Financial Intelligence [114.33879935277217]
  We introduce Herculean, the first skilled benchmark for agentic financial intelligence spanning four representative skill sets.<n>Across frontier agents, we find agents perform relatively well on Trading and Market Insights, but struggle substantially on Hedging and Auditing.<n>Overall, our results point to a key gap in current agents in turning financial reasoning into dependable workflow execution.
  arXiv  Detail & Related papers  (2026-05-14T04:30:49Z)
• Evaluating LLMs in Finance Requires Explicit Bias Consideration [88.38155218924999]
  Finance-specific biases can inflate performance, contaminate backtests, and make reported results useless for deployment claims.<n>No single bias is discussed in more than 28 percent of studies.<n>We propose a Structural Validity Framework and an evaluation checklist with minimal requirements for bias diagnosis and future system design.
  arXiv  Detail & Related papers  (2026-02-15T17:02:01Z)
• PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning [18.32501228579171]
  Professional Reasoning Bench (PRBench) is a realistic, open-ended, and difficult benchmark of real-world problems in Finance and Law.<n>We open-source its 1,100 expert-authored tasks and 19,356 expert-curated criteria, making it the largest public, rubric-based benchmark for both legal and finance domains.
  arXiv  Detail & Related papers  (2025-11-14T18:55:12Z)
• FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [57.43420753842626]
  FinLFQA is a benchmark designed to evaluate the ability of Large Language Models to generate long-form answers to complex financial questions.<n>We provide an automatic evaluation framework covering both answer quality and attribution quality.
  arXiv  Detail & Related papers  (2025-10-07T20:06:15Z)
• FinReflectKG - EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation [0.0]
  FinReflectKG - EvalBench is a benchmark and evaluation framework for financial knowledge graphs.<n>It links audited triples to source chunks from S&P 100 filings and supports single-pass, multi-pass, and reflection-agent-based extraction modes.<n>Our findings suggest that, when equipped with explicit bias controls, LLM-as-Judge protocols provide a reliable and cost-efficient alternative to human annotation.
  arXiv  Detail & Related papers  (2025-10-07T09:22:48Z)
• FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering [57.18367828883773]
  FinAgentBench is a benchmark for evaluating agentic retrieval with multi-step reasoning in finance.<n>The benchmark consists of 26K expert-annotated examples on S&P-500 listed firms.<n>We evaluate a suite of state-of-the-art models and demonstrate how targeted fine-tuning can significantly improve agentic retrieval performance.
  arXiv  Detail & Related papers  (2025-08-07T22:15:22Z)
• FinMaster: A Holistic Benchmark for Mastering Full-Pipeline Financial Workflows with LLMs [15.230256296815565]
  FinMaster is a benchmark designed to assess the capabilities of large language models (LLMs) in financial literacy, accounting, auditing, and consulting.<n>FinMaster comprises three main modules: FinSim, FinSuite, and FinEval.<n>Experiments reveal critical capability gaps in financial reasoning, with accuracy dropping from over 90% on basic tasks to merely 37% on complex scenarios.
  arXiv  Detail & Related papers  (2025-05-18T11:47:55Z)
• FinQA: A Dataset of Numerical Reasoning over Financial Data [52.7249610894623]
  We focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. We propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge.
  arXiv  Detail & Related papers  (2021-09-01T00:08:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.