HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research

• URL: http://arxiv.org/abs/2607.25151v1
• Date: Mon, 27 Jul 2026 23:50:46 GMT
• Title: HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research
• Abstract summary: We introduce HiEviDR-Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research.<n>HiEviDR-Bench represents each instance with an explicit evidence graph that captures evidence selection, cross-source linking, and aggregation.<n>We develop a traceability-oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness.
• Score: 52.87128503345243
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep research requires models to retrieve, connect, and synthesize evidence from large-scale heterogeneous sources to answer complex queries and produce analytical reports. Existing benchmarks mainly evaluate final outcomes, such as answer correctness, report quality, or citation alignment, while providing limited visibility into whether evidence is correctly selected, linked, and aggregated into supported claims and conclusions. To address this gap, we introduce HiEviDR-Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research. HiEviDR-Bench covers open-domain and academic-domain settings under both text-only and multimodal conditions, and represents each instance with an explicit evidence graph that captures evidence selection, cross-source linking, and aggregation from evidence to intermediate claims and final conclusions. Based on this formulation, we develop a traceability-oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness, together with a progressive gating mechanism for fine-grained error localization. HiEviDR-Bench contains 2,000 human-validated questions with evidence graphs across multiple difficulty levels. Experiments on 16 representative multimodal large language models show that, although many systems achieve strong report quality, their performance drops markedly on citation accuracy, claim construction, and answer correctness. Further analysis shows that the main bottlenecks lie in evidence identification and intermediate claim construction, revealing that strong surface-level report quality does not necessarily imply grounded multi-stage reasoning on our benchmark.

Related papers

• Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents [72.19038675107662]
  Mr. L is a benchmark for evaluating real-world deep research over long, irreducible chains of verifiable evidence.<n>Each question is constructed from a hidden Node-Relation graph and requires an average of 12.1 necessary intermediate conclusions.
  arXiv  Detail & Related papers  (2026-09-10T09:47:44Z)
• GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG [4.035350563640088]
  We present GraphLoom, a reliability-calibrated multimodal knowledge-graph RAG framework for compact and faithful evidence routing.<n>Instead of injecting all retrieved evidence into the generator, GraphLoom performs reliability-aware subgraph retrieval.<n>We show consistent gains in answer quality and evidence faithfulness over strong multimodal RAG, graph-retrieval, and open-source vision-language baselines.
  arXiv  Detail & Related papers  (2026-08-15T06:02:23Z)
• TVIR: Building Deep Research Agents Towards Text--Visual Interleaved Report Generation [63.89818340842003]
  TVIR (Text--Visual Interleaved Report Generation) is a benchmark of 100 expert-curated multimodal deep research tasks.<n>TVIR-Agent is a hierarchical multi-agent framework that serves as a strong baseline for constructing outlines and retrieving images.<n>We develop a dual-path evaluation framework that combines Textual Assessment and Visual Assessment.
  arXiv  Detail & Related papers  (2026-06-01T14:35:14Z)
• Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation [74.0621258662676]
  We propose Ptah, a multi-agent harness for interleaved report generation.<n>Ptah orchestrates the lifecycle from user query to rendered web report through planning, research, and writing stages.<n>A verifier agent serves as the harness's acceptance function, enforcing factual grounding, citation fidelity, and cross-modal consistency throughout the workflow.
  arXiv  Detail & Related papers  (2026-05-28T12:40:34Z)
• VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence [26.0945130521806]
  Existing benchmarks assess either textual answer correctness or pixel-level localization in isolation.<n>We introduce VISTAQA, a benchmark for joint evaluation of free-form answer correctness and pixel-level evidence grounding in visual question answering.<n>We introduce GROVE, a metric that enforces joint correctness by combining textual accuracy and grounding quality via a per-sample geometric mean.
  arXiv  Detail & Related papers  (2026-05-20T03:44:06Z)
• Stateful Evidence-Driven Retrieval-Augmented Generation with Iterative Reasoning [7.701766930531068]
  Stateful Evidence-Driven RAG with Iterative Reasoning is a framework that models question answering as a progressive evidence accumulation process.<n>The framework performs evidence-driven deficiency analysis to identify gaps and conflicts and iteratively refines queries to guide subsequent retrieval.<n> Experiments on multiple question answering benchmarks demonstrate consistent improvements over standard RAG and multi-step baselines.
  arXiv  Detail & Related papers  (2026-03-25T06:57:49Z)
• Multimodal Fact-Level Attribution for Verifiable Reasoning [80.60864342985748]
  Multimodal large language models (MLLMs) are increasingly used for real-world tasks involving multi-step reasoning and long-form generation.<n>Existing multimodal grounding benchmarks and evaluation methods fail to assess attribution in complex multimodal reasoning.<n>We introduce MuRGAt, a benchmark for evaluating fact-level multimodal attribution in settings that require reasoning beyond direct observation.
  arXiv  Detail & Related papers  (2026-02-12T03:10:02Z)
• MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering [51.19392014547221]
  Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures.<n>Current approaches rely on query-agnostic document representations that overlook salient content.<n>We propose the Multimodal Adaptive Retrieval-Augmented (MARA) framework, which introduces query-adaptive generation.
  arXiv  Detail & Related papers  (2026-02-01T12:27:40Z)
• MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents [37.98503734345155]
  We introduce MMDeepResearch-Bench (MMDR-Bench), a benchmark of 140 expert-crafted tasks across 21 domains.<n>Compared to prior setups, MMDR-Bench emphasizes report-style synthesis with explicit evidence use.<n>We propose a unified, interpretable evaluation pipeline: Formula-LLM Adaptive Evaluation (FLAE) for report quality, Trustworthy Retrieval-Aligned Citation Evaluation (TRACE) for citation-grounded evidence alignment, and Multimodal Support-Aligned Integrity Check (MOSAIC) for text-visual integrity.
  arXiv  Detail & Related papers  (2026-01-18T10:41:33Z)
• Unstructured Evidence Attribution for Long Context Query Focused Summarization [53.08341620504465]
  We propose to extract unstructured (i.e., spans of any length) evidence in order to acquire more relevant and consistent evidence than in the fixed granularity case.<n>We show how existing systems struggle to copy and properly cite unstructured evidence, which also tends to be "lost-in-the-middle"
  arXiv  Detail & Related papers  (2025-02-20T09:57:42Z)
• Unleashing Potential of Evidence in Knowledge-Intensive Dialogue Generation [37.29386687125705]
  We propose a framework to effectively incorporate Evidence in knowledge-Intensive Dialogue Generation (u-EIDG) Specifically, we introduce an automatic evidence generation framework that harnesses the power of Large Language Models (LLMs) to mine reliable evidence labels from unlabeled data. By utilizing these evidence labels, we train a reliable evidence indicator to effectively identify relevant evidence from retrieved passages.
  arXiv  Detail & Related papers  (2023-09-15T13:13:30Z)
• AmbiFC: Fact-Checking Ambiguous Claims with Evidence [57.7091560922174]
  We present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. We analyze disagreements arising from ambiguity when comparing claims against evidence in AmbiFC. We develop models for predicting veracity handling this ambiguity via soft labels.
  arXiv  Detail & Related papers  (2021-04-01T17:40:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.