S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing

• URL: http://arxiv.org/abs/2606.24441v1
• Date: Tue, 23 Jun 2026 11:19:34 GMT
• Title: S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing
• Abstract summary: We present an open-weight unified multimodal model for scientific image understanding, generation, and editing.<n>S1- Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B.<n>It supports scientific image understanding, generation, and editing in a unified framework.
• Score: 25.725824689793402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present S1-Omni-Image, an open-weight unified multimodal model for scientific image understanding, generation, and editing. Unlike general-purpose image generation models, scientific image tasks require not only high-fidelity synthesis, but also robust understanding of scientific semantics, structural relations, domain knowledge, and task intent. To this end, S1-Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B and couples its understanding capability with an image generation module under a unified think-before-generate paradigm. Given a user instruction, the model first produces a task-oriented reasoning trace, a textual answer, and a task special token; their hidden states are then injected into the generation module to condition image generation or editing. S1-Omni-Image supports scientific image understanding, generation, and editing in a unified framework. For generation, it focuses on scientific illustrations and text rendering, including logical diagrams, relational comparisons, data charts, and realistic scientific visualizations. For editing, it casts segmentation and other domain-specific vision tasks as native image editing problems, enabling multi-turn illustration editing, medical and geographic image segmentation, medical image translation, and scientific image super-resolution. We construct SciGenEdit, a 314K-sample training dataset, and release the model weights, inference code, and SciGenEdit-10K. Experiments show that S1-Omni-Image substantially improves scientific image generation and editing while preserving the scientific image understanding capability inherited from S1-VL-32B. It outperforms open-source models on GenExam and TechImage-Bench, achieves state-of-the-art results on four editing benchmarks including MSD, cigRockSEM, SynthRAD2025, and IXI, and maintains stable performance on scientific image understanding evaluations.

Related papers

• From Terminology to Diagrams: Visual-Instruction Generation for Scientific Diagram Understanding [0.0]
  Vision-language models (VLMs) have demonstrated strong performance in visual question answering with natural images.<n>We introduce a framework for generating large-scale diagram-grounded instruction data by leveraging terminology derived from scientific curricula.<n>We construct SciGram, a dataset of over 194K diagrams and 1.4M visual instructions across life, earth, and physical sciences.
  arXiv  Detail & Related papers  (2026-09-01T09:07:31Z)
• S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation [31.901449470705618]
  We present S1- Omni, a unified multimodal reasoning model for scientific understanding, prediction, and generation.<n>The architecture of S1- Omni is built upon three core components: unified representation of scientific data, natural-world knowledge alignment, and decoding for domain-specific tasks.<n>S1- Omni is trained on S1- Omni-Corpus, which covers 200 scientific tasks and contains millions of reasoning samples, and is evaluated on over 60 scientific benchmarks.
  arXiv  Detail & Related papers  (2026-07-17T06:56:08Z)
• DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editing [65.66783293569955]
  We introduce DisciplineGen-1M, a million-scale dataset that supports text-to-image generation and image editing.<n>It contains 1.2M samples spanning mathematics, physics, chemistry, biology, geography, computer science, economics, history, music, and sports.<n>Building on DisciplineGen-1M, we introduce a discipline-informed reasoning-generation model for both text-to-image generation and image editing.
  arXiv  Detail & Related papers  (2026-07-02T15:07:47Z)
• SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation [20.904891491567664]
  We introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation.<n>We elaborately create SciIR-82k, a large-scale dataset containing over 80,000 high-quality scientific image-text pairs from cutting-edge publications.<n>For evaluation, we propose SciIR-Bench, which aligns with these three semiotic levels and employs an Atomic Checklist to convert the outcome-oriented scientific accuracy into process-oriented, verifiable, fine-grained questions.
  arXiv  Detail & Related papers  (2026-06-29T10:59:10Z)
• Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models [49.93401412148884]
  We introduce FEPBench, a benchmark built from carefully selected high-quality scientific illustrations.<n>We evaluate T2I models along three dimensions: instruction faithfulness, reasoning enrichment, and semantic precision.<n>Results show that even state-of-the-art closed-source models, such as GPT Image 2 and Nano Banana Pro, still suffer from text-rendering bottlenecks, limited reasoning enrichment, and difficulty balancing generation richness with precision.
  arXiv  Detail & Related papers  (2026-06-04T09:49:02Z)
• HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer [104.09730595701468]
  We present HiDream-O1-Image, a unified generative foundation model via pixel-space Diffusion Transformer.<n>By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs.<n>Experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization.
  arXiv  Detail & Related papers  (2026-05-11T17:59:09Z)
• SIQA: Toward Reliable Scientific Image Quality Assessment [72.41803245808924]
  We introduce Scientific Image Quality Assessment (SIQA), a framework that models scientific image quality along two complementary dimensions.<n>We design two evaluation protocols: SIQA-U (Understanding), which measures semantic comprehension of scientific content through multiple-choice tasks, and SIQA-S (Scoring), which evaluates alignment with expert quality judgments.<n>Experiments across representative multimodal large language models (MLLMs) reveal a consistent discrepancy between scoring alignment and scientific understanding.
  arXiv  Detail & Related papers  (2026-03-05T06:57:26Z)
• UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing [44.071171929398076]
  multimodal models often struggle with complex synthesis tasks that demand deep reasoning.<n>We propose UniReason, a unified framework that harmonizes text-to-image generation and image editing.<n>We support this framework by systematically constructing a large-scale reasoning-centric dataset.
  arXiv  Detail & Related papers  (2026-02-02T18:34:35Z)
• Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility [57.83550091882176]
  We study scientific image synthesis across generation paradigms, evaluation, and downstream use.<n>We introduce SciGenBench, which evaluates generated images based on information utility and logical validity.<n>We show that fine-tuning Large Multimodal Models on rigorously verified synthetic scientific images yields consistent reasoning gains.
  arXiv  Detail & Related papers  (2026-01-17T14:18:36Z)
• S1-MMAlign: A Large-Scale, Multi-Disciplinary Dataset for Scientific Figure-Text Understanding [16.351123624587384]
  S1-MMAlign is a large-scale, multi-disciplinary multimodal dataset comprising over 15.5 million high-quality image-text pairs.<n>We introduce an AI-ready semantic enhancement pipeline that utilizes the Qwen-VL multimodal large model series to recaption images by synthesizing context from paper abstracts and citation contexts.
  arXiv  Detail & Related papers  (2026-01-01T08:54:51Z)
• Charts Are Not Images: On the Challenges of Scientific Chart Editing [66.38730113476677]
  textitFigEdit is a benchmark for scientific figure editing comprising over 30,000 samples.<n>Our benchmark demonstrates the profound limitations of pixel-level manipulation.<n>By releasing textitFigEdit, we aim to enable systematic progress in structure-aware figure editing.
  arXiv  Detail & Related papers  (2025-11-30T06:13:48Z)
• SridBench: Benchmark of Scientific Research Illustration Drawing of Image Generation Model [21.81341169834812]
  SridBench is the first benchmark for scientific figure generation.<n>It comprises 1,120 instances from leading scientific papers across 13 natural and computer science disciplines.<n>Results reveal that even top-tier models like GPT-4o-image lag behind human performance.
  arXiv  Detail & Related papers  (2025-05-28T08:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.