Towards Knowledgeable Deep Research: Framework and Benchmark

• URL: http://arxiv.org/abs/2604.07720v2
• Date: Fri, 10 Apr 2026 16:24:42 GMT
• Title: Towards Knowledgeable Deep Research: Framework and Benchmark
• Abstract summary: This paper introduces Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge.<n>We propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that integrates texts, figures, and tables into coherent multimodal reports.<n> Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics.
• Score: 105.87257445617922
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep Research (DR) requires LLM agents to autonomously perform multi-step information seeking, processing, and reasoning to generate comprehensive reports. In contrast to existing studies that mainly focus on unstructured web content, a more challenging DR task should additionally utilize structured knowledge to provide a solid data foundation, facilitate quantitative computation, and lead to in-depth analyses. In this paper, we refer to this novel task as Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge. Furthermore, we propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that reasons over both kinds of knowledge and integrates the texts, figures, and tables into coherent multimodal reports. The key design is the Structured Knowledge Analyzer, which utilizes both coding and vision-language models to produce figures, tables, and corresponding insights. To support systematic evaluation, we construct KDR-Bench, which covers 9 domains, includes 41 expert-level questions, and incorporates a large number of structured knowledge resources (e.g., 1,252 tables). We further annotate the main conclusions and key points for each question and propose three categories of evaluation metrics including general-purpose, knowledge-centric, and vision-enhanced ones. Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics, and even surpasses the Gemini DR agent on vision-enhanced metrics, highlighting its effectiveness in deep, structure-aware knowledge analysis. Finally, we hope this work can serve as a new foundation for structured knowledge analysis in DR agents and facilitate future multimodal DR studies.

Related papers

• S1-DeepResearch: Beyond Search, Toward Real-World Long-Horizon Research Agents [22.248115680330496]
  We propose a unified trajectory construction paradigm for deep research agents.<n>The proposed framework consists of graph-grounded task formulation, agentic trajectory rollout, and multi-dimensional trajectory verification.<n>S1-DeepResearch-32B achieves state-of-the-art performance among open-source models of comparable scale.
  arXiv  Detail & Related papers  (2026-06-13T16:04:42Z)
• TVIR: Building Deep Research Agents Towards Text--Visual Interleaved Report Generation [63.89818340842003]
  TVIR (Text--Visual Interleaved Report Generation) is a benchmark of 100 expert-curated multimodal deep research tasks.<n>TVIR-Agent is a hierarchical multi-agent framework that serves as a strong baseline for constructing outlines and retrieving images.<n>We develop a dual-path evaluation framework that combines Textual Assessment and Visual Assessment.
  arXiv  Detail & Related papers  (2026-06-01T14:35:14Z)
• Dr.Mi-Bench: A Modular-integrated Benchmark for Scientific Deep Research Agent [52.876617746453995]
  Dr.Mi-Bench is a Modular-integrated benchmark for scientific deep research (DR) agents.<n>Dr.Mi-Eval is a novel modular-integrated evaluation paradigm.
  arXiv  Detail & Related papers  (2025-11-30T17:16:47Z)
• Deep Research: A Systematic Survey [118.82795024422722]
  Deep Research (DR) aims to combine the reasoning capabilities of large language models with external tools, such as search engines.<n>This survey presents a comprehensive and systematic overview of deep research systems.
  arXiv  Detail & Related papers  (2025-11-24T15:28:28Z)
• ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents [11.666923792025313]
  Deep Research (DR) is an emerging agent application that leverages large language models to address open-ended queries.<n>We introduce ResearchRubrics, a standardized benchmark for DR built with over 2,800+ hours of human labor.<n>We also propose a new complexity framework for categorizing DR tasks along three axes: conceptual breadth, logical nesting, and exploration.
  arXiv  Detail & Related papers  (2025-11-10T23:07:14Z)
• Understanding DeepResearch via Reports [41.60038455664918]
  DeepResearch is a transformative AI paradigm, conducting expert-level research through sophisticated reasoning and multi-tool integration.<n> evaluating these systems remains critically challenging due to open-ended research scenarios and existing benchmarks that focus on isolated capabilities.<n>We introduce DeepResearch-ReportEval, a comprehensive framework designed to assess DeepResearch systems through their most representative outputs: research reports.
  arXiv  Detail & Related papers  (2025-10-09T07:03:43Z)
• A Rigorous Benchmark with Multidimensional Evaluation for Deep Research Agents: From Answers to Reports [24.09178055088843]
  Deep Research Agents (DRAs) exhibit the capabilities for task decomposition, cross-source retrieval, multi-stage reasoning, and structured output.<n>This paper introduces a rigorous benchmark and a multidimensional evaluation framework tailored to DRAs and report-style responses.<n>The framework enables comprehensive evaluation of long-form reports generated by DRAs, incorporating integrated scoring metrics for semantic quality, topical focus, and retrieval trustworthiness.
  arXiv  Detail & Related papers  (2025-10-02T16:40:02Z)
• KnowCoder-V2: Deep Knowledge Analysis [64.63893361811968]
  We propose a textbfKnowledgeable textbfDeep textbfResearch (textbfKDR) framework that empowers deep research with deep knowledge analysis capability.<n>It introduces an independent knowledge organization phase to preprocess large-scale, domain-relevant data into systematic knowledge offline.<n>It then extends deep research with an additional kind of reasoning steps that perform complex knowledge computation in an online manner.
  arXiv  Detail & Related papers  (2025-06-07T18:01:25Z)
• StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization [94.31508613367296]
  Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) We propose StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure. Experiments show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios.
  arXiv  Detail & Related papers  (2024-10-11T13:52:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.