WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning

• URL: http://arxiv.org/abs/2607.09328v1
• Date: Fri, 10 Jul 2026 12:09:21 GMT
• Title: WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning
• Abstract summary: Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages.<n>This source-internal evidence integration is central to real-world long-document analysis, yet existing benchmarks largely sidestep it.<n>We introduce WILDTRACE, a benchmark of 481 tasks over 214 naturally occurring long-form sources.
• Score: 63.9071746889252
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages. In an incident report, the operating condition, design flaw, and missed safety check that jointly explain a disaster may appear dozens of sections apart; in a novel, a character's true motive may surface only through scenes far removed from the moment it becomes relevant. This source-internal evidence integration is central to real-world long-document analysis, yet existing benchmarks largely sidestep it. Needle probes, planted facts, and reverse-engineered multi-hop chains embed evidence that may differ from the host text in distribution, placement, or register, making it unclear whether strong performance reflects genuine source reasoning or distributional artifacts. We introduce WILDTRACE, a benchmark of 481 tasks over 214 naturally occurring long-form sources such as technical incident reports and lesser-known literary narratives, where all evidence trails arise from the document's own causal, temporal, and narrative logic. Drawing on Pearl's causal hierarchy and prior multi-hop reasoning typologies, we define seven source-internal evidence geometries that characterize the distinct relational demands of analytical reading in long documents. A source-first construction pipeline mines candidate trails from document structure before writing questions; each item then undergoes multi-stage validation covering clue necessity, answer groundedness, rubric fidelity, contamination resistance and answerability. As models are increasingly entrusted with real-world high-stakes analytical tasks, this gap between accessing information and reasoning over naturally dispersed evidence emerges as a defining challenge for the next stage of long-context research.

Related papers

• Navigating Sparse Evidence: Agentic Visual RAG via Explicit Context Selection and Consolidation [50.92223196290564]
  We propose a unified agent loop for explicit evidence selection and consolidation.<n>During exploration, SCoRE retains only query-relevant observations and their source pointers in a maintained textual ledger.<n>At termination, it reloads the referenced original images and consolidates the visual evidence for answering, arranging it into a logical sequence.
  arXiv  Detail & Related papers  (2026-09-14T16:11:35Z)
• HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research [52.87128503345243]
  We introduce HiEviDR-Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research.<n>HiEviDR-Bench represents each instance with an explicit evidence graph that captures evidence selection, cross-source linking, and aggregation.<n>We develop a traceability-oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness.
  arXiv  Detail & Related papers  (2026-07-27T23:50:46Z)
• DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments [51.049999642138]
  We introduce DRNOISE, a 100-task benchmark for answer recovery under misleading evidence.<n>Each task has a unique gold answer supported by two corroborating indirect record chains; the paired noisy condition adds one plausible document that states a conflicting answer directly.<n>Across agents with strong clean-task performance, this single intervention causes 66-88 percentage-point accuracy drops.
  arXiv  Detail & Related papers  (2026-07-19T15:20:40Z)
• Building Agent Harnesses for Scientific Curation from Multimodal Sources [17.12384426105798]
  We study scientific curation from multimodal sources and introduce Beaver, an agent harness that extracts structured information from scientific papers.<n>We show that Beaver reaches 81.0 on Gold-Referenced Attribute Score (GRAS), an attribute-level measure of agreement with gold curated records, outperforming frontier agents by over 23 absolute points.
  arXiv  Detail & Related papers  (2026-06-19T00:36:34Z)
• Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation [74.0621258662676]
  We propose Ptah, a multi-agent harness for interleaved report generation.<n>Ptah orchestrates the lifecycle from user query to rendered web report through planning, research, and writing stages.<n>A verifier agent serves as the harness's acceptance function, enforcing factual grounding, citation fidelity, and cross-modal consistency throughout the workflow.
  arXiv  Detail & Related papers  (2026-05-28T12:40:34Z)
• Skill-Contracted Agents for Evidence-Aware Materials Literature Analysis [8.055788977592144]
  Materials science literature analysis requires simultaneous attention to composition, processing, characterization, and property relationships.<n>Here we present AlphaAgent, a skill-driven agent framework that decouples retrieval-based question answering from paper-level report generation.<n>A dedicated skill rewrites user requests into material-specific intents, queries a curated index of more than 300,000 papers from the Journal Reports Citation and Metallurgical Engineering category, and reformulates queries when initial evidence is insufficient.
  arXiv  Detail & Related papers  (2026-05-10T13:00:32Z)
• Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities [32.76303717104482]
  We introduce DeR2, a controlled deep-research sandbox that isolates document-grounded reasoning.<n>DeR2 decouples evidence access from reasoning via four regimes--Instruction-only, Concepts, Related-only, and Full-set.<n> Experiments across a diverse set of state-of-the-art foundation models reveal substantial variation and significant headroom.
  arXiv  Detail & Related papers  (2026-01-29T16:26:19Z)
• WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Research [73.58638285105971]
  This paper tackles textbfopen-ended deep research (OEDR), a complex challenge where AI agents must synthesize vast web-scale information into insightful reports.<n>We introduce textbfWebWeaver, a novel dual-agent framework that emulates the human research process.<n>Our framework establishes a new state-of-the-art across major OEDR benchmarks, including DeepResearch Bench, DeepConsult, and DeepResearchGym.
  arXiv  Detail & Related papers  (2025-09-16T17:57:21Z)
• Graph-based Retrieval for Claim Verification over Cross-Document Evidence [0.6853165736531939]
  We conjecture that a graph-based approach can be beneficial to identify fragmented evidence. We tested this hypothesis by building, over the whole corpus, a large graph that interconnects text portions by means of mentioned entities. Our experiments show that leveraging on a graph structure is beneficial in identifying a reasonably small portion of passages related to a claim.
  arXiv  Detail & Related papers  (2021-09-13T14:54:26Z)
• Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification [19.130541561303293]
  We propose a novel topic-aware evidence reasoning and stance-aware aggregation model for fact verification. Tests conducted on two benchmark datasets demonstrate the superiority of the proposed model over several state-of-the-art approaches for fact verification.
  arXiv  Detail & Related papers  (2021-06-02T14:33:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.