Multi-Agent Debate and Visual Information Extraction for SeePhys Pro: A 1st-Place Technical Report from ICML 2026 AI4Math Track 3 Challenge

• URL: http://arxiv.org/abs/2607.21946v1
• Date: Fri, 24 Jul 2026 03:44:30 GMT
• Title: Multi-Agent Debate and Visual Information Extraction for SeePhys Pro: A 1st-Place Technical Report from ICML 2026 AI4Math Track 3 Challenge
• Abstract summary: This report presents our approach to the Challenge Track3: SeePhys Pro at the 3rd AI for Math Workshop.<n>The task is to answer college-level physics questions whose statement and figure may be given partly or entirely as an image.<n>We address the task with a two-stage framework: a visual information extraction stage that re-expresses figure content as solver-readable text, and a reasoning stage that orchestrates three heterogeneous solvers.
• Score: 26.80223000446421
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This technical report presents our approach to Challenge Track~3: SeePhys Pro at the 3rd AI for Math Workshop, where the task is to answer college-level physics questions whose statement and figure may be given partly or entirely as an image. Visual physics problems become substantially harder for large language models when the decisive information resides in a figure rather than in the text, and this modality gap widens as more of the problem migrates into the image. We address the task with a two-stage framework: a visual information extraction stage that re-expresses figure content as solver-readable text to close the modality gap, and a reasoning stage that orchestrates three heterogeneous solvers through multi-agent debate. Our analysis yields two findings: the gain from orchestration comes from reliable answer selection rather than from additional debate, and the value of a figure aid scales with how much of the problem is locked inside the image. The resulting pipeline improves overall accuracy over a single-agent baseline from 0.643 to 0.802 on the public split, and won 1st place on both the public and the private leaderboard (private overall 0.743).

Related papers

• PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback [101.62446136707372]
  We present a benchmark for multi-turn diagram generation containing 292 images annotated with 3,518 user requirements.<n>PaperBananaBench identifies unsatisfied requirements and converts k of them into natural language feedback.<n>We introduce PaperBanana-Interact, a multi-agent system that refines diagrams via an internal critique-and-refine loop.
  arXiv  Detail & Related papers  (2026-08-31T04:52:32Z)
• PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems? [3.7188197739781903]
  PhysElite is a large-scale bilingual benchmark for Olympiad-level physics reasoning.<n>For each problem, we provide corresponding visual diagrams, step-by-step bilingual Chinese-English solution derivations, and the final answer.<n>We benchmark 18 open-source and closed-source MLLMs, and find that even the strongest model reaches only 33.7% answer accuracy.
  arXiv  Detail & Related papers  (2026-08-25T19:48:42Z)
• NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1) [115.35873876159131]
  We present an overview of the NTIRE 2026 challenge on the 3rd Restore Any Image Model in the Wild, specifically focusing on Track 1.<n>In total, the challenge attracted nearly 200 registrations and over 2,500 submissions.<n>The top-performing methods significantly advanced the state of the art in professional IQA.
  arXiv  Detail & Related papers  (2026-04-14T09:44:35Z)
• MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generation [37.74857807039394]
  We introduce MathGen, a benchmark of 900 problems spanning seven core domains.<n>We show that even the best closed-source model reaches only 42.0% overall accuracy.<n>Overall, current T2I models remain far from competent at even elementary mathematical visual generation.
  arXiv  Detail & Related papers  (2026-03-30T02:22:35Z)
• InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search [48.79494320593913]
  We introduce O3-Bench, a new benchmark designed to evaluate multimodal reasoning with interleaved attention to visual details.<n>O3-Bench features challenging problems that require agents to piece together subtle visual information from distinct image areas through multi-step reasoning.<n>We propose InSight-o3, a multi-agent framework consisting of a visual reasoning agent (vReasoner) and a visual search agent (vSearcher)
  arXiv  Detail & Related papers  (2025-12-21T14:23:07Z)
• Multimodal Reasoning for Science: Technical Report and 1st Place Solution to the ICML 2025 SeePhys Challenge [13.14311042732615]
  We introduce a caption-assisted reasoning framework that effectively bridges visual and textual modalities.<n>Our approach achieved 1st place in the ICML 2025 AI for Math Workshop & Challenge 2: SeePhys, highlighting its effectiveness and robustness.
  arXiv  Detail & Related papers  (2025-09-07T14:47:32Z)
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models [48.99485386990197]
  We conduct a case study using a synthetic dataset solvable only through visual reasoning.<n>We then introduce ChartMuseum, a new Chart Question Answering (QA) benchmark containing 1,162 expert-annotated questions.<n>Although humans achieve 93% accuracy, the best-performing model Gemini-2.5-Pro attains only 63.0%, and the leading open-source LVLM Qwen2.5-VL-72B-Instruct achieves only 38.5%.
  arXiv  Detail & Related papers  (2025-05-19T17:59:27Z)
• Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks [62.758680527838436]
  We propose Leopard, an MLLM tailored for handling vision-language tasks involving multiple text-rich images.<n>First, we curated about one million high-quality multimodal instruction-tuning data, tailored to text-rich, multi-image scenarios.<n>Second, we proposed an adaptive high-resolution multi-image encoding module to dynamically optimize the allocation of visual sequence length.
  arXiv  Detail & Related papers  (2024-10-02T16:55:01Z)
• Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination [40.83905569501714]
  We leverage a single Vision-Language Model (VLM) to generate a structured representation of the question using HTML, then render the HTML as an image, and finally use the same VLM to answer the question using both the question and the image. We evaluate our approach on three mathematics tasks and nine general-purpose reasoning tasks using state-of-the-art (LLAVA-1.5 and GEMINI PRO) VLMs.
  arXiv  Detail & Related papers  (2024-01-16T00:46:29Z)
• More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problems [0.4660328753262075]
  We evaluate the performance of two large multimodal models on visual assignments. GPT-4V solved 96.7% of these visual problems, struggling minimally with a single Parsons problem. Bard performed poorly by only solving 69.2% of problems, struggling with common issues like hallucinations and refusals.
  arXiv  Detail & Related papers  (2023-11-03T14:47:17Z)
• PV2TEA: Patching Visual Modality to Textual-Established Information Extraction [59.76117533540496]
  We patch the visual modality to the textual-established attribute information extractor. PV2TEA is an encoder-decoder architecture equipped with three bias reduction schemes. Empirical results on real-world e-Commerce datasets demonstrate up to 11.74% absolute (20.97% relatively) F1 increase over unimodal baselines.
  arXiv  Detail & Related papers  (2023-06-01T05:39:45Z)
• MERLOT: Multimodal Neural Script Knowledge Models [74.05631672657452]
  We introduce MERLOT, a model that learns multimodal script knowledge by watching millions of YouTube videos with transcribed speech. MERLOT exhibits strong out-of-the-box representations of temporal commonsense, and achieves state-of-the-art performance on 12 different video QA datasets. On Visual Commonsense Reasoning, MERLOT answers questions correctly with 80.6% accuracy, outperforming state-of-the-art models of similar size by over 3%.
  arXiv  Detail & Related papers  (2021-06-04T17:57:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.