BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows

• URL: http://arxiv.org/abs/2604.11304v1
• Date: Mon, 13 Apr 2026 11:02:32 GMT
• Title: BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows
• Abstract summary: Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional.<n>To evaluate frontier AI agents in a high-value, labor-intensive profession, we introduce BankerToolBench (BTB)<n>BTB is an open-source benchmark of end-to-end analytical deliverables routinely performed by junior investment bankers.
• Score: 12.076413424824679
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional workflows. To evaluate frontier AI agents in a high-value, labor-intensive profession, we introduce BankerToolBench (BTB): an open-source benchmark of end-to-end analytical workflows routinely performed by junior investment bankers. To develop an ecologically valid benchmark grounded in representative work environments, we collaborated with 502 investment bankers from leading firms. BTB requires agents to execute senior banker requests by navigating data rooms, using industry tools (market data platform, SEC filings database), and generating multi-file deliverables--including Excel financial models, PowerPoint pitch decks, and PDF/Word reports. Completing a BTB task takes bankers up to 21 hours, underscoring the economic stakes of successfully delegating this work to AI. BTB enables automated evaluation of any LLM or agent, scoring deliverables against 100+ rubric criteria defined by veteran investment bankers to capture stakeholder utility. Testing 9 frontier models, we find that even the best-performing model (GPT-5.4) fails nearly half of the rubric criteria and bankers rate 0% of its outputs as client-ready. Our failure analysis reveals key obstacles (such as breakdowns in cross-artifact consistency) and improvement directions for agentic AI in high-stakes professional workflows.

Related papers

• StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [63.57092302641323]
  We introduce textbfStartupBench, an E2E agent benchmark grounded in market-validated AI startup products.<n>We study AI products with demonstrated adoption, together with their product and users, to identify real-world tasks for which AI has established practical demand.<n>Our results reveal that many market-validated agents remain beyond the reliable capabilities of current general-purpose agents.
  arXiv  Detail & Related papers  (2026-08-18T14:01:32Z)
• Agents' Last Exam [275.79071391137074]
  Agents' Last Exam (ALE) is a benchmark designed to evaluate AI agents on long horizon, economically valuable, real world tasks with verifiable outcomes.<n>Developed in collaboration with 250+ industry experts, ALE covers non-physical industries defined with reference to O*NET / SOC 2018.<n>Current results show that the hardest tier remains far from saturated: across mainstream harness and backbone configurations, the average full pass rate is below 1%.
  arXiv  Detail & Related papers  (2026-06-03T20:20:46Z)
• Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning [0.13999481573773073]
  Hedge-Bench 1.0 is a benchmark of 102 actual, on-the-job tasks grounded in the explicit reasoning traces of professional hedge fund analysts.<n>We publish the dataset and evaluation harness at Frontier models.com/Trata-Inc/trata-hedge-bench.
  arXiv  Detail & Related papers  (2026-06-02T17:11:56Z)
• Herculean: An Agentic Benchmark for Financial Intelligence [114.33879935277217]
  We introduce Herculean, the first skilled benchmark for agentic financial intelligence spanning four representative skill sets.<n>Across frontier agents, we find agents perform relatively well on Trading and Market Insights, but struggle substantially on Hedging and Auditing.<n>Overall, our results point to a key gap in current agents in turning financial reasoning into dependable workflow execution.
  arXiv  Detail & Related papers  (2026-05-14T04:30:49Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language World Models [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces [65.11019654023978]
  LongCLI-Bench is a benchmark designed to evaluate agentic capabilities across long-horizon, realistic tasks.<n>We curated 20 high-quality, long-horizon tasks from over 1,000 computer science assignments and real-world tasks.<n>Experiments reveal that even state-of-the-art agents achieve pass rates below 20% in LongCLI-Bench.
  arXiv  Detail & Related papers  (2026-02-15T23:12:57Z)
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces [126.23612941699565]
  Terminal-Bench 2.0 is a benchmark composed of 89 tasks in computer terminal environments inspired by problems from real world.<n>We show that frontier models and agents score less than 65% on the benchmark.<n>We publish the dataset and evaluation harness to assist developers and researchers in future work at https://www.tbench.ai/.
  arXiv  Detail & Related papers  (2026-01-17T01:29:30Z)
• UpBench: A Dynamically Evolving Real-World Labor-Market Agentic Benchmark Framework Built for Human-Centric AI [2.0619484032730813]
  UpBench is a benchmark grounded in real jobs drawn from the global Upwork labor marketplace.<n>Each task corresponds to a verified client transaction, anchoring evaluation in genuine work activity and financial outcomes.<n>UpBench employs a rubric-based evaluation framework, in which expert freelancers decompose each job into detailed, verifiable acceptance criteria and assess AI submissions with per-criterion feedback.
  arXiv  Detail & Related papers  (2025-11-15T17:39:37Z)
• Towards Outcome-Oriented, Task-Agnostic Evaluation of AI Agents [1.0305173936249623]
  This white paper proposes a novel framework of eleven outcome-based, task-agnostic performance metrics for AI agents.<n>We introduce metrics such as Goal Completion Rate (GCR), Autonomy Index (AIx), Multi-Step Task Resilience (MTR), and Business Impact Efficiency (BIE)<n>Our results reveal significant performance trade-offs between different agent designs, highlighting the Hybrid Agent as the most consistently high-performing model.
  arXiv  Detail & Related papers  (2025-11-11T13:40:46Z)
• Anti-Money Laundering Machine Learning Pipelines; A Technical Analysis on Identifying High-risk Bank Clients with Supervised Learning [1.149137041907724]
  We propose a comprehensive and systematic approach for developing machine learning pipelines to identify high-risk bank clients.<n>The dataset included 195,789 IDs and we employed a 16-step customer design and statistical analysis.<n>Our pipeline achieved a mean area under the receiver operating characteristic curve (AUROC) of 0.961 with a standard deviation (SD) of 0.005.
  arXiv  Detail & Related papers  (2025-09-11T03:53:10Z)
• FinGAIA: A Chinese Benchmark for AI Agents in Real-World Financial Domain [17.682962451679312]
  FinGAIA comprises 407 meticulously crafted tasks, spanning seven major financial sub-domains.<n>The best-performing agent, ChatGPT, achieved an overall accuracy of 48.9%, which, while superior to non-professionals, still lags financial experts by over 35 percentage points.
  arXiv  Detail & Related papers  (2025-07-23T04:19:16Z)
• Establishing Best Practices for Building Rigorous Agentic Benchmarks [94.69724201080155]
  We show that many agentic benchmarks have issues in task setup or reward design.<n>Such issues can lead to under- or overestimation of agents' performance by up to 100% in relative terms.<n>We introduce the Agentic Benchmark Checklist (ABC), a set of guidelines that we synthesized from our benchmark-building experience.
  arXiv  Detail & Related papers  (2025-07-03T17:35:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.