SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation

• URL: http://arxiv.org/abs/2606.30124v1
• Date: Mon, 29 Jun 2026 10:59:10 GMT
• Title: SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation
• Abstract summary: We introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation.<n>We elaborately create SciIR-82k, a large-scale dataset containing over 80,000 high-quality scientific image-text pairs from cutting-edge publications.<n>For evaluation, we propose SciIR-Bench, which aligns with these three semiotic levels and employs an Atomic Checklist to convert the outcome-oriented scientific accuracy into process-oriented, verifiable, fine-grained questions.
• Score: 20.904891491567664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Text-to-Image (T2I) models have shown remarkable success in generating photorealistic visual content, they still struggle with the rigorous semantic alignment and logical reasoning required for scientific imagery. Inspired by Peirce's Semiotic Triad, we introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation. We formalize scientific reasoning into three core dimensions: Entity Structure (Icon), Scientific Process (Index), and Scientific Law (Symbol). Specifically, to overcome the scarcity of training data in scientific image generation, we elaborately create SciIR-82k, a large-scale dataset containing over 80,000 high-quality scientific image-text pairs from cutting-edge publications. The dataset is hierarchically organized according to the semiotic dimensions and incorporates a Scientific Reasoning Chain-of-Thought (Sci-RCoT) to explicitly model underlying visual logic. For evaluation, we propose SciIR-Bench, which aligns with these three semiotic levels and employs an Atomic Checklist to convert the outcome-oriented scientific accuracy into process-oriented, verifiable, fine-grained questions. Our extensive experiments reveal significant deficiencies in current models' scientific reasoning capabilities. Furthermore, by fine-tuning on the SciIR-82k dataset, we developed the Qwen-Image-SciIR model, which achieves a substantial improvement on the SciIR-Bench, increasing the final score from 35\% to 43\%, laying a solid foundation for future advances in scientific image generation.

Related papers

• DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editing [65.66783293569955]
  We introduce DisciplineGen-1M, a million-scale dataset that supports text-to-image generation and image editing.<n>It contains 1.2M samples spanning mathematics, physics, chemistry, biology, geography, computer science, economics, history, music, and sports.<n>Building on DisciplineGen-1M, we introduce a discipline-informed reasoning-generation model for both text-to-image generation and image editing.
  arXiv  Detail & Related papers  (2026-07-02T15:07:47Z)
• S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing [25.725824689793402]
  We present an open-weight unified multimodal model for scientific image understanding, generation, and editing.<n>S1- Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B.<n>It supports scientific image understanding, generation, and editing in a unified framework.
  arXiv  Detail & Related papers  (2026-06-23T11:19:34Z)
• Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models [49.93401412148884]
  We introduce FEPBench, a benchmark built from carefully selected high-quality scientific illustrations.<n>We evaluate T2I models along three dimensions: instruction faithfulness, reasoning enrichment, and semantic precision.<n>Results show that even state-of-the-art closed-source models, such as GPT Image 2 and Nano Banana Pro, still suffer from text-rendering bottlenecks, limited reasoning enrichment, and difficulty balancing generation richness with precision.
  arXiv  Detail & Related papers  (2026-06-04T09:49:02Z)
• SIQA: Toward Reliable Scientific Image Quality Assessment [72.41803245808924]
  We introduce Scientific Image Quality Assessment (SIQA), a framework that models scientific image quality along two complementary dimensions.<n>We design two evaluation protocols: SIQA-U (Understanding), which measures semantic comprehension of scientific content through multiple-choice tasks, and SIQA-S (Scoring), which evaluates alignment with expert quality judgments.<n>Experiments across representative multimodal large language models (MLLMs) reveal a consistent discrepancy between scoring alignment and scientific understanding.
  arXiv  Detail & Related papers  (2026-03-05T06:57:26Z)
• Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility [57.83550091882176]
  We study scientific image synthesis across generation paradigms, evaluation, and downstream use.<n>We introduce SciGenBench, which evaluates generated images based on information utility and logical validity.<n>We show that fine-tuning Large Multimodal Models on rigorously verified synthetic scientific images yields consistent reasoning gains.
  arXiv  Detail & Related papers  (2026-01-17T14:18:36Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Graphics4Science: Computer Graphics for Scientific Impacts [69.54528197718207]
  This course explores the relationship between computer graphics and science.<n>We show how core methods, such as geometric reasoning and physical modeling, provide inductive biases that help address challenges in both fields.<n>We aim to reframe graphics as a modeling language for science by bridging vocabulary gaps between the two communities.
  arXiv  Detail & Related papers  (2025-06-18T18:06:58Z)
• SridBench: Benchmark of Scientific Research Illustration Drawing of Image Generation Model [21.81341169834812]
  SridBench is the first benchmark for scientific figure generation.<n>It comprises 1,120 instances from leading scientific papers across 13 natural and computer science disciplines.<n>Results reveal that even top-tier models like GPT-4o-image lag behind human performance.
  arXiv  Detail & Related papers  (2025-05-28T08:51:01Z)
• Science-T2I: Addressing Scientific Illusions in Image Synthesis [29.376938884546764]
  We introduce Science-T2I, an expert-annotated adversarial dataset comprising adversarial 20k image pairs with 9k prompts.<n>We present SciScore, an end-to-end reward model that refines the assessment of generated images based on scientific knowledge.
  arXiv  Detail & Related papers  (2025-04-17T17:44:19Z)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models [57.96527452844273]
  We introduce SciInstruct, a suite of scientific instructions for training scientific language models capable of college-level scientific reasoning. We curated a diverse and high-quality dataset encompassing physics, chemistry, math, and formal proofs. To verify the effectiveness of SciInstruct, we fine-tuned different language models with SciInstruct, i.e., ChatGLM3 (6B and 32B), Llama3-8B-Instruct, and Mistral-7B: MetaMath.
  arXiv  Detail & Related papers  (2024-01-15T20:22:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.