SciMDR: Benchmarking and Advancing Scientific Multimodal Document Reasoning

• URL: http://arxiv.org/abs/2603.12249v1
• Date: Thu, 12 Mar 2026 17:57:52 GMT
• Title: SciMDR: Benchmarking and Advancing Scientific Multimodal Document Reasoning
• Abstract summary: We introduce the synthesize synthesize framework, which generates faithful, isolated QA pairs on focused segments.<n>We construct SciMDR, a large-scale training dataset for crossmodal comprehension.<n>Experiments demonstrate that SciMDR models achieve significant improvements across multiple scientific QA benchmarks.
• Score: 59.93568326270048
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Constructing scientific multimodal document reasoning datasets for foundation model training involves an inherent trade-off among scale, faithfulness, and realism. To address this challenge, we introduce the synthesize-and-reground framework, a two-stage pipeline comprising: (1) Claim-Centric QA Synthesis, which generates faithful, isolated QA pairs and reasoning on focused segments, and (2) Document-Scale Regrounding, which programmatically re-embeds these pairs into full-document tasks to ensure realistic complexity. Using this framework, we construct SciMDR, a large-scale training dataset for cross-modal comprehension, comprising 300K QA pairs with explicit reasoning chains across 20K scientific papers. We further construct SciMDR-Eval, an expert-annotated benchmark to evaluate multimodal comprehension within full-length scientific workflows. Experiments demonstrate that models fine-tuned on SciMDR achieve significant improvements across multiple scientific QA benchmarks, particularly in those tasks requiring complex document-level reasoning.

Related papers

• Agents-K1: Towards Agent-native Knowledge Orchestration [59.22667222947257]
  textbfAgents-K1 is an end-to-end knowledge orchestration pipeline.<n>It converts raw documents into agent-native scientific knowledge graphs.<n>We process 2.46 million scientific papers across six subjects to produce textbfScholar-KG, of which we release a one-million-paper subset.
  arXiv  Detail & Related papers  (2026-06-11T17:58:35Z)
• PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers [52.97586643334384]
  We introduce PaperScope, a benchmark for agentic deep research.<n>It is built on a knowledge graph of over 2,000 AI papers spanning three years.<n>It integrates semantically related key information nodes and employs optimized random-walk article selector.
  arXiv  Detail & Related papers  (2026-04-13T11:07:08Z)
• Paper2SysArch: Structure-Constrained System Architecture Generation from Scientific Papers [10.395280181257737]
  We introduce a novel benchmark to quantitatively evaluate the automated generation of diagrams from text.<n>It consists of 3,000 research papers paired with their corresponding high-quality ground-truth diagrams and is accompanied by a three-tiered evaluation metric.<n>We propose Paper2Arch, an end-to-end system that leverages multi-agent collaboration to convert papers into structured, editable diagrams.
  arXiv  Detail & Related papers  (2025-11-22T12:24:30Z)
• Discourse-Aware Scientific Paper Recommendation via QA-Style Summarization and Multi-Level Contrastive Learning [2.105564340986074]
  OMRC-MR is a hierarchical framework that integrates QA-style OMRC summarization, multi-level contrastive learning, and structure-aware re-ranking for scholarly recommendation.<n> Experiments on DBLP, S2ORC, and the newly constructed Sci-OMRC dataset demonstrate that OMRC-MR consistently surpasses state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-05T09:55:12Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding [100.29587871213624]
  We introduce DocR1, an MLLM trained with a novel RL framework, Evidence Page-Guided GRPO.<n>EviGRPO incorporates an evidence-aware reward mechanism that promotes a coarse-to-fine reasoning strategy.<n>We show that DocR1 achieves state-of-the-art performance on multi-page tasks, while consistently maintaining strong results on single-page benchmarks.
  arXiv  Detail & Related papers  (2025-08-10T12:03:45Z)
• MMESGBench: Pioneering Multimodal Understanding and Complex Reasoning Benchmark for ESG Tasks [56.350173737493215]
  Environmental, Social, and Governance (ESG) reports are essential for evaluating sustainability practices, ensuring regulatory compliance, and promoting financial transparency.<n>MMESGBench is a first-of-its-kind benchmark dataset to evaluate multimodal understanding and complex reasoning across structurally diverse and multi-source ESG documents.<n>MMESGBench comprises 933 validated QA pairs derived from 45 ESG documents, spanning across seven distinct document types and three major ESG source categories.
  arXiv  Detail & Related papers  (2025-07-25T03:58:07Z)
• M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models [27.910693214922052]
  We introduce M3SciQA, a multi-modal, multi-document scientific question answering benchmark for evaluating foundation models. M3SciQA consists of 1,452 expert-annotated questions spanning 70 natural language processing paper clusters. Our results indicate that current foundation models still significantly underperform compared to human experts in multi-modal information retrieval and in reasoning across multiple scientific documents.
  arXiv  Detail & Related papers  (2024-11-06T17:52:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.