FinTradeBench: A Financial Reasoning Benchmark for LLMs

• URL: http://arxiv.org/abs/2603.19225v2
• Date: Fri, 20 Mar 2026 19:02:41 GMT
• Title: FinTradeBench: A Financial Reasoning Benchmark for LLMs
• Abstract summary: FinTradeBench is a benchmark for evaluating financial reasoning that integrates company fundamentals and trading signals.<n>FinTradeBench contains 1,400 questions grounded in NASDAQ-100 companies over a ten-year historical window.
• Score: 3.2342681547638796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world financial decision-making is a challenging problem that requires reasoning over heterogeneous signals, including company fundamentals derived from regulatory filings and trading signals computed from price dynamics. Recently, with the advancement of Large Language Models (LLMs), financial analysts have begun to use them for financial decision-making tasks. However, existing financial question answering benchmarks for testing these models primarily focus on company balance sheet data and rarely evaluate reasoning over how company stocks trade in the market or their interactions with fundamentals. To take advantage of the strengths of both approaches, we introduce FinTradeBench, a benchmark for evaluating financial reasoning that integrates company fundamentals and trading signals. FinTradeBench contains 1,400 questions grounded in NASDAQ-100 companies over a ten-year historical window. The benchmark is organized into three reasoning categories: fundamentals-focused, trading-signal-focused, and hybrid questions requiring cross-signal reasoning. To ensure reliability at scale, we adopt a calibration-then-scaling framework that combines expert seed questions, multi-model response generation, intra-model self-filtering, numerical auditing, and human-LLM judge alignment. We evaluate 14 LLMs under zero-shot prompting and retrieval-augmented settings and witness a clear performance gap. Retrieval substantially improves reasoning over textual fundamentals, but provides limited benefit for trading-signal reasoning. These findings highlight fundamental challenges in the numerical and time-series reasoning for current LLMs and motivate future research in financial intelligence.

Related papers

• FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [57.43420753842626]
  FinLFQA is a benchmark designed to evaluate the ability of Large Language Models to generate long-form answers to complex financial questions.<n>We provide an automatic evaluation framework covering both answer quality and attribution quality.
  arXiv  Detail & Related papers  (2025-10-07T20:06:15Z)
• StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets? [44.10622904101254]
  Large language models (LLMs) have recently demonstrated strong capabilities as autonomous agents.<n>We introduce StockBench, a benchmark designed to evaluate LLM agents in realistic, multi-month stock trading environments.<n>Our evaluation shows that while most LLM agents struggle to outperform the simple buy-and-hold baseline, several models demonstrate the potential to deliver higher returns and manage risk more effectively.
  arXiv  Detail & Related papers  (2025-10-02T16:54:57Z)
• Trading-R1: Financial Trading with LLM Reasoning via Reinforcement Learning [19.52468210547666]
  Trading-R1 is a financially-aware model that incorporates strategic thinking and planning for comprehensive thesis composition, facts-grounded analysis, and volatility-adjusted decision making.<n>The system generates structured, evidence-based investment theses that support disciplined and interpretable trading decisions.
  arXiv  Detail & Related papers  (2025-09-14T20:13:41Z)
• Your AI, Not Your View: The Bias of LLMs in Investment Analysis [62.388554963415906]
  In finance, Large Language Models (LLMs) face frequent knowledge conflicts arising from discrepancies between their pre-trained parametric knowledge and real-time market data.<n>These conflicts are especially problematic in real-world investment services, where a model's inherent biases can misalign with institutional objectives.<n>We propose an experimental framework to investigate emergent behaviors in such conflict scenarios, offering a quantitative analysis of bias in investment analysis.
  arXiv  Detail & Related papers  (2025-07-28T16:09:38Z)
• FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning [82.7292329605713]
  FinChain is the first benchmark specifically designed for verifiable Chain-of-Thought evaluation in finance.<n>It spans 58 topics across 12 financial domains, each represented by parameterized symbolic templates with executable Python traces.<n>FinChain exposes persistent weaknesses in multi-step financial reasoning and provides a foundation for developing trustworthy, interpretable, and verifiable financial AI.
  arXiv  Detail & Related papers  (2025-06-03T06:44:42Z)
• Deriving Strategic Market Insights with Large Language Models: A Benchmark for Forward Counterfactual Generation [55.2788567621326]
  We introduce a novel benchmark, FIN-FORCE-FINancial FORward Counterfactual Evaluation.<n>By curating financial news headlines, FIN-FORCE supports LLM based forward counterfactual generation.<n>This paves the way for scalable and automated solutions for exploring and anticipating future market developments.
  arXiv  Detail & Related papers  (2025-05-26T02:41:50Z)
• Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation [5.892346584607669]
  We propose FinAR-Bench, a benchmark dataset focusing on financial statement analysis.<n>We break this task into three measurable steps: extracting key information, calculating financial indicators, and applying logical reasoning.<n>Our findings offer a clear understanding of LLMs current strengths and limitations in fundamental analysis.
  arXiv  Detail & Related papers  (2025-05-22T07:06:20Z)
• Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance [35.617409883103335]
  FinReason is the first financial reasoning benchmark covering multi-table analysis, long-context reasoning, and equation-based tasks.<n>We introduce FinCoT, the first open high-fidelity CoT corpus for finance, distilled from seven QA datasets.<n>We develop Fin-o1, the first open financial reasoning models trained via supervised fine-tuning and GRPO-based RL.
  arXiv  Detail & Related papers  (2025-02-12T05:13:04Z)
• FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models [0.0]
  FinanceQA is a testing suite that evaluates LLMs' performance on complex numerical financial analysis tasks that mirror real-world investment work.<n>Current LLMs fail to meet the strict accuracy requirements of financial institutions, with models failing approximately 60% of realistic tasks.<n>Results show that higher-quality training data is needed to support such tasks, which we experiment with using OpenAI's fine-tuning API.
  arXiv  Detail & Related papers  (2025-01-30T00:06:55Z)
• FinBen: A Holistic Financial Benchmark for Large Language Models [75.09474986283394]
  FinBen is the first extensive open-source evaluation benchmark, including 36 datasets spanning 24 financial tasks. FinBen offers several key innovations: a broader range of tasks and datasets, the first evaluation of stock trading, novel agent and Retrieval-Augmented Generation (RAG) evaluation, and three novel open-source evaluation datasets for text summarization, question answering, and stock trading.
  arXiv  Detail & Related papers  (2024-02-20T02:16:16Z)
• PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance [63.51545277822702]
  PIXIU is a comprehensive framework including the first financial large language model (LLMs) based on fine-tuning LLaMA with instruction data. We propose FinMA by fine-tuning LLaMA with the constructed dataset to be able to follow instructions for various financial tasks. We conduct a detailed analysis of FinMA and several existing LLMs, uncovering their strengths and weaknesses in handling critical financial tasks.
  arXiv  Detail & Related papers  (2023-06-08T14:20:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.