E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning

• URL: http://arxiv.org/abs/2609.14302v1
• Date: Sun, 13 Sep 2026 05:50:05 GMT
• Title: E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning
• Abstract summary: We introduce E2A-Bench, a benchmark for financial chart reasoning, constructed from 323 HS300 constituents.<n>E2A-Bench evaluates grounding, reasoning-action consistency, evidence-confidence calibration, and directional coverage.<n>We show that financial VLM evaluation should trace the full evidence-to-action chain rather than rely on a single hallucination score.
• Score: 3.81869701471217
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Can financial vision-language models (VLMs) turn chart evidence into reliable action recommendations? Existing hallucination evaluations are mostly claim-centric; they assess whether generated statements are supported, but not whether evidence remains traceable through rationale, confidence, and final action. We introduce E2A-Bench, a 969-query benchmark for financial chart reasoning, constructed from 323 HS300 constituents under three input modalities with deterministic OHLCV-derived evidence anchors. E2A-Bench evaluates grounding, reasoning-action consistency, evidence-confidence calibration, and directional coverage through UCR, RCI, ECI, and NDR, where NDR measures coverage-aware evidence-to-action reliability rather than realized trading performance. Evaluating 20 VLMs reveals three failures hidden by scalar hallucination scores: the lowest-UCR model ranks near the bottom by NDR due to only 6.4% directional coverage; oracle-aided verification reduces unsupported claims but can collapse coverage; and financial fine-tuning amplifies the BUY:SELL ratio by factors of 4.21 to 4.68 across strict base-fine-tuned pairs. These results show that financial VLM evaluation should trace the full evidence-to-action chain rather than rely on a single hallucination score. Code and data: https://github.com/wanng-ide/E2A-Bench

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysis [42.62413276719373]
  A trustworthy result should be supported by a valid path from the user question to the relevant data evidence.<n>This requirement creates two diagnostic questions: whether an LLM can refuse to answer or ask for clarification when such a path does not exist, and whether it can preserve the correct analysis when the same evidence is expressed in different table forms.<n>We introduce TrustDABench, a benchmark that operationalizes these questions as reliability and robustness.
  arXiv  Detail & Related papers  (2026-08-25T07:09:01Z)
• From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers [15.44881677475537]
  Large language models (LLMs) are increasingly used to provide prior causal knowledge for structural causal discovery.<n>We evaluate 12 instruction-tuned open-weight models across six benchmark causal graphs, five prompting strategies, and four confidence sources.<n>Our results suggest LLMs are better viewed as sources of externally validated soft causal priors than as direct evidence of causal structure.
  arXiv  Detail & Related papers  (2026-08-24T15:55:47Z)
• Trust but Verify: Introducing DAVinCI -- A Framework for Dual Attribution and Verification in Claim Inference for Language Models [53.279391576560755]
  Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across a wide range of NLP tasks, yet they remain prone to factual inaccuracies and limitation.<n>This poses significant risks in high-stakes domains such as healthcare, law, and scientific communication, where trust and verifiability are paramount.<n>We introduce DAVinCI - a Dual Attribution and Verification framework designed to enhance the factual reliability and interpretability of LLM outputs.
  arXiv  Detail & Related papers  (2026-04-23T01:37:50Z)
• FinReflectKG -- HalluBench: GraphRAG Hallucination Benchmark for Financial Question Answering Systems [0.0]
  Current Knowledge Graph (KG)-augmented QA systems lack systematic mechanisms to detect hallucinations.<n>We introduce FinBench-QA-Hallucination, a benchmark for evaluating hallucination detection methods in KG-augmented financial QA over SEC 10-K filings.<n>Our findings highlight vulnerabilities in current KG-augmented systems and provide insights for building reliable financial information systems.
  arXiv  Detail & Related papers  (2026-03-11T04:37:53Z)
• Evaluating LLMs in Finance Requires Explicit Bias Consideration [88.38155218924999]
  Finance-specific biases can inflate performance, contaminate backtests, and make reported results useless for deployment claims.<n>No single bias is discussed in more than 28 percent of studies.<n>We propose a Structural Validity Framework and an evaluation checklist with minimal requirements for bias diagnosis and future system design.
  arXiv  Detail & Related papers  (2026-02-15T17:02:01Z)
• All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection [67.89888669159899]
  RFC Bench is a benchmark for evaluating large language models on financial misinformation under realistic news.<n>The benchmark defines two complementary tasks: reference free misinformation detection and comparison based diagnosis.
  arXiv  Detail & Related papers  (2026-01-07T18:18:28Z)
• Look As You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning [55.232400251303794]
  Look As You Think (LAT) is a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution.<n>LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5.
  arXiv  Detail & Related papers  (2025-11-15T02:50:23Z)
• FinVet: A Collaborative Framework of RAG and External Fact-Checking Agents for Financial Misinformation Detection [0.0]
  FinVet is a novel framework that integrates two Retrieval-Augmented Generation pipelines with external fact-checking.<n>It provides evidence-backed verdicts, source attribution, confidence scores, and explicit uncertainty flags when evidence is insufficient.
  arXiv  Detail & Related papers  (2025-10-13T17:31:49Z)
• Reasoning Models Are More Easily Gaslighted Than You Think [85.84943447589511]
  We evaluate three state-of-the-art reasoning models, including OpenAI's o4-mini, Claude-3.7-Sonnet and Gemini-2.5-Flash.<n>Our evaluation reveals significant accuracy drops following gaslighting negation prompts.<n>We introduce GaslightingBench-R, a new diagnostic benchmark designed to evaluate reasoning models' susceptibility to defend their belief.
  arXiv  Detail & Related papers  (2025-06-11T12:52:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.