EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents

• URL: http://arxiv.org/abs/2604.05557v1
• Date: Tue, 07 Apr 2026 07:58:55 GMT
• Title: EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents
• Abstract summary: We introduce EpiBench, an episodic multi-turn benchmark that instantiates short research.<n>Given a research task, agents must navigate across papers over multiple turns, align evidence from figures and tables, and use the accumulated evidence in the memory to answer objective questions.<n>Our experiments show that even the leading model achieves an accuracy of only 29.23% on the hard split, indicating substantial room for improvement.
• Score: 38.845762856105175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific research follows multi-turn, multi-step workflows that require proactively searching the literature, consulting figures and tables, and integrating evidence across papers to align experimental settings and support reproducible conclusions. This joint capability is not systematically assessed in existing benchmarks, which largely under-evaluate proactive search, multi-evidence integration and sustained evidence use over time. In this work, we introduce EpiBench, an episodic multi-turn multimodal benchmark that instantiates short research workflows. Given a research task, agents must navigate across papers over multiple turns, align evidence from figures and tables, and use the accumulated evidence in the memory to answer objective questions that require cross paper comparisons and multi-figure integration. EpiBench introduces a process-level evaluation framework for fine-grained testing and diagnosis of research agents. Our experiments show that even the leading model achieves an accuracy of only 29.23% on the hard split, indicating substantial room for improvement in multi-turn, multi-evidence research workflows, providing an evaluation platform for verifiable and reproducible research agents.

Related papers

• Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents [72.19038675107662]
  Mr. L is a benchmark for evaluating real-world deep research over long, irreducible chains of verifiable evidence.<n>Each question is constructed from a hidden Node-Relation graph and requires an average of 12.1 necessary intermediate conclusions.
  arXiv  Detail & Related papers  (2026-09-10T09:47:44Z)
• PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers [52.97586643334384]
  We introduce PaperScope, a benchmark for agentic deep research.<n>It is built on a knowledge graph of over 2,000 AI papers spanning three years.<n>It integrates semantically related key information nodes and employs optimized random-walk article selector.
  arXiv  Detail & Related papers  (2026-04-13T11:07:08Z)
• MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents [37.98503734345155]
  We introduce MMDeepResearch-Bench (MMDR-Bench), a benchmark of 140 expert-crafted tasks across 21 domains.<n>Compared to prior setups, MMDR-Bench emphasizes report-style synthesis with explicit evidence use.<n>We propose a unified, interpretable evaluation pipeline: Formula-LLM Adaptive Evaluation (FLAE) for report quality, Trustworthy Retrieval-Aligned Citation Evaluation (TRACE) for citation-grounded evidence alignment, and Multimodal Support-Aligned Integrity Check (MOSAIC) for text-visual integrity.
  arXiv  Detail & Related papers  (2026-01-18T10:41:33Z)
• Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning [112.16686518063456]
  We introduce textbfMulti-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time.<n>MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making.<n>Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67% over a multi-agent baseline, and by 8.67% over comparable single-agent baselines
  arXiv  Detail & Related papers  (2026-01-14T17:57:43Z)
• Format Matters: The Robustness of Multimodal LLMs in Reviewing Evidence from Tables and Charts [19.571644726057666]
  We design and conduct experiments to assess the ability of multimodal large language models to verify scientific claims using both tables and charts as evidence.<n>Using this adapted dataset, we evaluate 12 multimodal LLMs and find that current models perform better with table-based evidence while struggling with chart-based evidence.<n>Our analysis also reveals that smaller multimodal LLMs (under 8B) show weak correlation in performance between table-based and chart-based tasks, indicating limited cross-modal generalization.
  arXiv  Detail & Related papers  (2025-11-13T08:29:47Z)
• Learning to Summarize by Learning to Quiz: Adversarial Agentic Collaboration for Long Document Summarization [86.98098988779809]
  We propose SummQ, a novel adversarial multi-agent framework for long document summarization.<n>Our approach employs summary generators and reviewers that work collaboratively to create and evaluate comprehensive summaries.<n>We evaluate SummQ on three widely used long document summarization benchmarks.
  arXiv  Detail & Related papers  (2025-09-25T08:36:19Z)
• Progressive Multimodal Reasoning via Active Retrieval [64.74746997923967]
  Multi-step multimodal reasoning tasks pose significant challenges for large language models (MLLMs)<n>We propose AR-MCTS, a universal framework designed to progressively improve the reasoning capabilities of MLLMs.<n>We show that AR-MCTS can optimize sampling diversity and accuracy, yielding reliable multimodal reasoning.
  arXiv  Detail & Related papers  (2024-12-19T13:25:39Z)
• MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking [0.283600654802951]
  We present a summarization model designed to generate claim-specific summaries useful for fact-checking from multimodal datasets. We introduce a dynamic perceiver-based model that can handle inputs from multiple modalities of arbitrary lengths. Our approach outperforms the SOTA approach by 4.6% in the claim verification task on the MOCHEG dataset.
  arXiv  Detail & Related papers  (2024-07-18T01:33:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.