DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections

• URL: http://arxiv.org/abs/2508.15851v1
• Date: Wed, 20 Aug 2025 08:17:45 GMT
• Title: DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections
• Authors: Jiwon Park, Seohyun Pyeon, Jinwoo Kim, Rina Carines Cabal, Yihao Ding, Soyeon Caren Han,
• Abstract summary: We propose DocHop-QA, a large-scale benchmark for multimodal, multi-document, multi-hop question answering.<n> DocHop-QA is domain-agnostic and incorporates diverse information formats, including textual passages, tables, and structural layout cues.<n>We evaluate DocHop-QA through four tasks spanning structured index prediction, generative answering, and multimodal integration.
• Score: 23.428084176322866
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite recent advances in large language models (LLMs), most QA benchmarks are still confined to single-paragraph or single-document settings, failing to capture the complexity of real-world information-seeking tasks. Practical QA often requires multi-hop reasoning over information distributed across multiple documents, modalities, and structural formats. Although prior datasets made progress in this area, they rely heavily on Wikipedia-based content and unimodal plain text, with shallow reasoning paths that typically produce brief phrase-level or single-sentence answers, thus limiting their realism and generalizability. We propose DocHop-QA, a large-scale benchmark comprising 11,379 QA instances for multimodal, multi-document, multi-hop question answering. Constructed from publicly available scientific documents sourced from PubMed, DocHop-QA is domain-agnostic and incorporates diverse information formats, including textual passages, tables, and structural layout cues. Unlike existing datasets, DocHop-QA does not rely on explicitly hyperlinked documents; instead, it supports open-ended reasoning through semantic similarity and layout-aware evidence synthesis. To scale realistic QA construction, we designed an LLM-driven pipeline grounded in 11 high-frequency scientific question concepts. We evaluated DocHop-QA through four tasks spanning structured index prediction, generative answering, and multimodal integration, reflecting both discriminative and generative paradigms. These tasks demonstrate DocHop-QA's capacity to support complex, multimodal reasoning across multiple documents.

Related papers

• OIDA-QA: A Multimodal Benchmark for Analyzing the Opioid Industry Documents Archive [50.468138755368805]
  Opioid crisis represents a significant moment in public health.<n>Data and documents disclosed in the UCSF-JHU Opioid Industry Documents Archive (OIDA)<n>In this paper, we tackle this challenge by organizing the original dataset according to document attributes.
  arXiv  Detail & Related papers  (2025-11-13T03:27:32Z)
• Doc-Researcher: A Unified System for Multimodal Document Parsing and Deep Research [31.973886754355547]
  Doc-Researcher is a unified system that bridges the gap between text-only, vision-only, and hybrid paradigms.<n>We introduce M4DocBench, the first benchmark for Multi-modal, Multi-hop, Multi-document, and Multi-turn deep research.<n>Experiments demonstrate Doc-Researcher achieves 50.6% accuracy, 3.4xbetter than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-10-24T16:07:54Z)
• MMESGBench: Pioneering Multimodal Understanding and Complex Reasoning Benchmark for ESG Tasks [56.350173737493215]
  Environmental, Social, and Governance (ESG) reports are essential for evaluating sustainability practices, ensuring regulatory compliance, and promoting financial transparency.<n>MMESGBench is a first-of-its-kind benchmark dataset to evaluate multimodal understanding and complex reasoning across structurally diverse and multi-source ESG documents.<n>MMESGBench comprises 933 validated QA pairs derived from 45 ESG documents, spanning across seven distinct document types and three major ESG source categories.
  arXiv  Detail & Related papers  (2025-07-25T03:58:07Z)
• Docopilot: Improving Multimodal Models for Document-Level Understanding [87.60020625241178]
  We present a high-quality document-level dataset, Doc-750K, designed to support in-depth understanding of multimodal documents.<n>This dataset includes diverse document structures, extensive cross-page dependencies, and real question-answer pairs derived from the original documents.<n>Building on the dataset, we develop a native multimodal model, Docopilot, which can accurately handle document-level dependencies without relying on RAG.
  arXiv  Detail & Related papers  (2025-07-19T16:03:34Z)
• M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding [63.33447665725129]
  We introduce M3DocRAG, a novel multi-modal RAG framework that flexibly accommodates various document contexts. M3DocRAG can efficiently handle single or many documents while preserving visual information. We also present M3DocVQA, a new benchmark for evaluating open-domain DocVQA over 3,000+ PDF documents with 40,000+ pages.
  arXiv  Detail & Related papers  (2024-11-07T18:29:38Z)
• PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering [13.625303311724757]
  Document Question Answering (QA) presents a challenge in understanding visually-rich documents (VRD) We propose PDF-MVQA, which is tailored for research journal articles, encompassing multiple pages and multimodal information retrieval.
  arXiv  Detail & Related papers  (2024-04-19T09:00:05Z)
• Read and Think: An Efficient Step-wise Multimodal Language Model for Document Understanding and Reasoning [0.0]
  Existing document understanding models tend to generate answers with a single word or phrase directly. We use Multi-modal Large Language Models (MLLMs) to generate step-wise question-and-answer pairs for document images. We then use the generated high-quality data to train a humanized document understanding and reasoning model, dubbed DocAssistant.
  arXiv  Detail & Related papers  (2024-02-26T01:17:50Z)
• PDFTriage: Question Answering over Long, Structured Documents [60.96667912964659]
  Representing structured documents as plain text is incongruous with the user's mental model of these documents with rich structure. We propose PDFTriage that enables models to retrieve the context based on either structure or content. Our benchmark dataset consists of 900+ human-generated questions over 80 structured documents.
  arXiv  Detail & Related papers  (2023-09-16T04:29:05Z)
• MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data [7.063167712310221]
  Existing question answering benchmarks over hybrid data only include a single flat table in each document. We construct a new large-scale benchmark, MultiHiertt, with QA pairs over Multi Hierarchical Tabular and Textual data. Results show that MultiHiertt presents a strong challenge for existing baselines whose results lag far behind the performance of human experts.
  arXiv  Detail & Related papers  (2022-06-03T00:24:35Z)
• End-to-End Multihop Retrieval for Compositional Question Answering over Long Documents [93.55268936974971]
  We propose a multi-hop retrieval method, DocHopper, to answer compositional questions over long documents. At each step, DocHopper retrieves a paragraph or sentence embedding from the document, mixes the retrieved result with the query, and updates the query for the next step. We demonstrate that utilizing document structure in this was can largely improve question-answering and retrieval performance on long documents.
  arXiv  Detail & Related papers  (2021-06-01T03:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.