Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows

• URL: http://arxiv.org/abs/2608.24842v1
• Date: Tue, 25 Aug 2026 17:31:25 GMT
• Title: Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows
• Abstract summary: Large language models (LLMs) are increasingly deployed as AI analysts to process financial disclosures.<n>We identify a retrieval-integration gap in long-context financial analysis.<n>We find that a risk disclosure's influence on investment judgments falls to the experimental noise floor.
• Score: 7.410179371973137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly deployed as AI analysts to process financial disclosures and support AI-assisted investment decisions. Yet such systems are usually evaluated by what they can retrieve, not whether retrieved information affects their judgments. We identify a retrieval-integration gap in long-context financial analysis. Holding focal-firm information fixed and varying only unrelated context from 2,000 to 128,000 tokens, we find that a risk disclosure's influence on investment judgments falls to the experimental noise floor even as direct retrieval remains accurate. The pattern replicates across model families and judgment tasks and in experiments removing real disclosures from actual 10-K filings. More capable models postpone but do not eliminate the gap. Causal memory interventions show that compressed summaries and source-text lookup jointly transmit disclosures into judgments. Workflow architecture determines whether this transmission succeeds: chunk-and-summarize pipelines evict relevant information, whereas a targeted, structured restatement adjacent to the decision restores its influence. AI analyst performance is therefore jointly determined by model capability and workflow architecture. Retrieval-based evaluations can certify systems whose investment judgments ignore information they demonstrably retrieved.

Related papers

• SENTRY: Deterministic, Intelligent Risk Assessment for IT Change Management [3.6992654014782516]
  This paper presents SENTRY, a risk assessment platform that replaces questionnaire-based scoring with a deterministic machine learning pipeline.<n>The system combines structured operational metadata, application dependency graphs, and historical incident records with a hybrid semantic and lexical search over historical change requests.<n>It achieves a ROC AUC of 0.87 and 85% overall accuracy, and it detects high-risk changes at roughly 3.25 times the rate of the existing process.
  arXiv  Detail & Related papers  (2026-08-21T15:18:21Z)
• Can Agentic Trading Systems Pay for Their Own Intelligence? [42.600136171993874]
  We introduce TradeLens, a trace-grounded diagnostic toolkit for evaluating agentic trading systems.<n>We conduct extensive analysis across backbone models, capital scales, trading frequencies, and system architectures.<n>Our results show that viability hinges on intelligence-to-profit conversion.
  arXiv  Detail & Related papers  (2026-07-11T12:31:54Z)
• When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysis [88.39399763902419]
  We show that financial compression can alter the investment judgment supported by the original source.<n>In agentic systems, such losses may recur across intermediate steps and amplify throughout the decision process.<n>We propose Agentic Context Compression, which generates multiple candidate compressions and audits their disagreements against the original source.
  arXiv  Detail & Related papers  (2026-06-28T07:44:54Z)
• Auditing Language Model Unlearning via Information Decomposition [68.48660428111593]
  We introduce an interpretable, information-theoretic framework for auditing unlearning using Partial Information Decomposition (PID)<n>By comparing model representations before and after unlearning, we decompose the mutual information with the forgotten data into distinct components, formalizing the notions of unlearned and residual knowledge.<n>Our work introduces a principled, representation-level audit for unlearning, offering theoretical insight and actionable tools for safer deployment of language models.
  arXiv  Detail & Related papers  (2026-01-21T15:51:19Z)
• Hallucination Detection and Mitigation in Large Language Models [0.0]
  Large Language Models (LLMs) and Large Reasoning Models (LRMs) offer transformative potential for high-stakes domains like finance and law.<n>Their tendency to hallucinate, generating factually incorrect or unsupported content, poses a critical reliability risk.<n>This paper introduces a comprehensive framework for hallucination management, built on a continuous improvement cycle driven by root cause awareness.
  arXiv  Detail & Related papers  (2026-01-14T23:19:37Z)
• Modeling Fairness in Recruitment AI via Information Flow [1.290382979353427]
  We apply an information flow-based modeling framework to a real-world recruitment process that integrates automated candidate matching with human decision-making.<n>We identify where biases may emerge, how they can propagate through the system, and what downstream impacts they may have on candidates.
  arXiv  Detail & Related papers  (2025-11-16T21:01:55Z)
• FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering [57.18367828883773]
  FinAgentBench is a benchmark for evaluating agentic retrieval with multi-step reasoning in finance.<n>The benchmark consists of 26K expert-annotated examples on S&P-500 listed firms.<n>We evaluate a suite of state-of-the-art models and demonstrate how targeted fine-tuning can significantly improve agentic retrieval performance.
  arXiv  Detail & Related papers  (2025-08-07T22:15:22Z)
• Leveraging Large Language Models for Tacit Knowledge Discovery in Organizational Contexts [0.4499833362998487]
  We propose an agent-based framework to iteratively reconstruct dataset descriptions through interactions with employees.<n>Our results show that the agent achieves 94.9% full-knowledge recall, with self-critical feedback scores strongly correlating with external literature critic scores.<n>These findings highlight the agent's ability to navigate organizational complexity and capture fragmented knowledge that would otherwise remain inaccessible.
  arXiv  Detail & Related papers  (2025-07-04T21:09:32Z)
• Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG [51.120170062795566]
  We propose Divide-Then-Align (DTA) to endow RAG systems with the ability to respond with "I don't know" when the query is out of the knowledge boundary.<n>DTA balances accuracy with appropriate abstention, enhancing the reliability and trustworthiness of retrieval-augmented systems.
  arXiv  Detail & Related papers  (2025-05-27T08:21:21Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.