Science-T2I: Addressing Scientific Illusions in Image Synthesis

• URL: http://arxiv.org/abs/2504.13129v1
• Date: Thu, 17 Apr 2025 17:44:19 GMT
• Title: Science-T2I: Addressing Scientific Illusions in Image Synthesis
• Abstract summary: We introduce Science-T2I, an expert-annotated adversarial dataset comprising adversarial 20k image pairs with 9k prompts.<n>We present SciScore, an end-to-end reward model that refines the assessment of generated images based on scientific knowledge.
• Score: 29.376938884546764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel approach to integrating scientific knowledge into generative models, enhancing their realism and consistency in image synthesis. First, we introduce Science-T2I, an expert-annotated adversarial dataset comprising adversarial 20k image pairs with 9k prompts, covering wide distinct scientific knowledge categories. Leveraging Science-T2I, we present SciScore, an end-to-end reward model that refines the assessment of generated images based on scientific knowledge, which is achieved by augmenting both the scientific comprehension and visual capabilities of pre-trained CLIP model. Additionally, based on SciScore, we propose a two-stage training framework, comprising a supervised fine-tuning phase and a masked online fine-tuning phase, to incorporate scientific knowledge into existing generative models. Through comprehensive experiments, we demonstrate the effectiveness of our framework in establishing new standards for evaluating the scientific realism of generated content. Specifically, SciScore attains performance comparable to human-level, demonstrating a 5% improvement similar to evaluations conducted by experienced human evaluators. Furthermore, by applying our proposed fine-tuning method to FLUX, we achieve a performance enhancement exceeding 50% on SciScore.

Related papers

• Sci-VBench: Evaluating Knowledge- and Reasoning-Intensive Video Generation in Science Domains [15.749662582403614]
  We introduce Sci-VBench, a benchmark for evaluating knowledge- and reasoning-intensive video generation across scientific domains.<n>It contains 1,253 expert-annotated examples spanning 60 subjects across four core disciplines: Natural Science, Healthcare, Humanities & Social Sciences, and Engineering.
  arXiv  Detail & Related papers  (2026-08-10T17:27:32Z)
• SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation [20.904891491567664]
  We introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation.<n>We elaborately create SciIR-82k, a large-scale dataset containing over 80,000 high-quality scientific image-text pairs from cutting-edge publications.<n>For evaluation, we propose SciIR-Bench, which aligns with these three semiotic levels and employs an Atomic Checklist to convert the outcome-oriented scientific accuracy into process-oriented, verifiable, fine-grained questions.
  arXiv  Detail & Related papers  (2026-06-29T10:59:10Z)
• Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs [8.700507644617964]
  Large language models (LLMs) are increasingly used to communicate and explain scientific concepts.<n>We evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base.
  arXiv  Detail & Related papers  (2026-06-19T12:03:02Z)
• Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models [49.93401412148884]
  We introduce FEPBench, a benchmark built from carefully selected high-quality scientific illustrations.<n>We evaluate T2I models along three dimensions: instruction faithfulness, reasoning enrichment, and semantic precision.<n>Results show that even state-of-the-art closed-source models, such as GPT Image 2 and Nano Banana Pro, still suffer from text-rendering bottlenecks, limited reasoning enrichment, and difficulty balancing generation richness with precision.
  arXiv  Detail & Related papers  (2026-06-04T09:49:02Z)
• SIQA: Toward Reliable Scientific Image Quality Assessment [72.41803245808924]
  We introduce Scientific Image Quality Assessment (SIQA), a framework that models scientific image quality along two complementary dimensions.<n>We design two evaluation protocols: SIQA-U (Understanding), which measures semantic comprehension of scientific content through multiple-choice tasks, and SIQA-S (Scoring), which evaluates alignment with expert quality judgments.<n>Experiments across representative multimodal large language models (MLLMs) reveal a consistent discrepancy between scoring alignment and scientific understanding.
  arXiv  Detail & Related papers  (2026-03-05T06:57:26Z)
• Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility [57.83550091882176]
  We study scientific image synthesis across generation paradigms, evaluation, and downstream use.<n>We introduce SciGenBench, which evaluates generated images based on information utility and logical validity.<n>We show that fine-tuning Large Multimodal Models on rigorously verified synthetic scientific images yields consistent reasoning gains.
  arXiv  Detail & Related papers  (2026-01-17T14:18:36Z)
• SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence [99.30934038146965]
  SciEvalKit focuses on the core competencies of scientific intelligence.<n>It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science.<n>The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.
  arXiv  Detail & Related papers  (2025-12-26T17:36:02Z)
• Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows [203.3527268311731]
  We present an operational SGI definition grounded in the Practical Inquiry Model (PIM)<n>We operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning.<n>Our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
  arXiv  Detail & Related papers  (2025-12-18T12:44:36Z)
• SciGPT: A Large Language Model for Scientific Literature Understanding and Knowledge Discovery [3.779883844533933]
  This paper presents SciGPT, a domain-adapted model for scientific literature understanding and ScienceBench, an open source benchmark tailored to evaluate scientific LLMs.<n>Built on the Qwen3 architecture, SciGPT incorporates three key innovations: (1) low-cost domain distillation via a two-stage pipeline to balance performance and efficiency; (2) a Sparse Mixture-of-Experts attention mechanism that cuts memory consumption by 55% for 32,000 long-token reasoning; and (3) knowledge-aware adaptation integrating domain-specific nuances.<n> Experimental results on ScienceBench show that SciGPT outperforms GPT-4o in core scientific tasks including sequence
  arXiv  Detail & Related papers  (2025-09-09T16:09:19Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Dynamic Knowledge Exchange and Dual-diversity Review: Concisely Unleashing the Potential of a Multi-Agent Research Team [53.38438460574943]
  IDVSCI is a multi-agent framework built on large language models (LLMs)<n>It incorporates two key innovations: a Dynamic Knowledge Exchange mechanism and a Dual-Diversity Review paradigm.<n>Results show that IDVSCI consistently achieves the best performance across two datasets.
  arXiv  Detail & Related papers  (2025-06-23T07:12:08Z)
• ScienceMeter: Tracking Scientific Knowledge Updates in Language Models [79.33626657942169]
  Large Language Models (LLMs) are increasingly used to support scientific research, but their knowledge of scientific advancements can quickly become outdated.<n>We introduce ScienceMeter, a new framework for evaluating scientific knowledge update methods over scientific knowledge spanning the past, present, and future.
  arXiv  Detail & Related papers  (2025-05-30T07:28:20Z)
• AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy [59.32718342798908]
  We introduce AstroVisBench, the first benchmark for both scientific computing and visualization in the astronomy domain.<n>We present an evaluation of state-of-the-art language models, showing a significant gap in their ability to engage in astronomy research as useful assistants.
  arXiv  Detail & Related papers  (2025-05-26T21:49:18Z)
• SciMantify -- A Hybrid Approach for the Evolving Semantification of Scientific Knowledge [0.4499833362998487]
  We propose an evolution model of knowledge representation, inspired by the 5-star Linked Open Data (LOD) model.<n>We develop a hybrid approach, called SciMantify, to support its evolving semantification.<n>We implement the approach in the Open Research Knowledge Graph (ORKG), an established platform for improving the findability, accessibility, interoperability, and reusability of scientific knowledge.
  arXiv  Detail & Related papers  (2025-04-14T07:57:55Z)
• BoxingGym: Benchmarking Progress in Automated Experimental Design and Model Discovery [24.630117520005257]
  We introduce BoxingGym, a benchmark with 10 environments for evaluating experimental design and model discovery. We compute the expected information gain (EIG), an information-theoretic quantity which measures how much an experiment reduces uncertainty about the parameters of a generative model. We find that current LLMs, such as GPT-4o, struggle with both experimental design and model discovery.
  arXiv  Detail & Related papers  (2025-01-02T21:15:57Z)
• Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System [62.832818186789545]
  Virtual Scientists (VirSci) is a multi-agent system designed to mimic the teamwork inherent in scientific research. VirSci organizes a team of agents to collaboratively generate, evaluate, and refine research ideas. We show that this multi-agent approach outperforms the state-of-the-art method in producing novel scientific ideas.
  arXiv  Detail & Related papers  (2024-10-12T07:16:22Z)
• SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding [22.131371019641417]
  Despite Large Language Models' success, they face challenges in scientific literature understanding.<n>We propose a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT)<n>We present a suite of LLMs: SciLitLLM, specialized in scientific literature understanding.
  arXiv  Detail & Related papers  (2024-08-28T05:41:52Z)
• SciDMT: A Large-Scale Corpus for Detecting Scientific Mentions [52.35520385083425]
  We present SciDMT, an enhanced and expanded corpus for scientific mention detection. The corpus consists of two components: 1) the SciDMT main corpus, which includes 48 thousand scientific articles with over 1.8 million weakly annotated mention annotations in the format of in-text span, and 2) an evaluation set, which comprises 100 scientific articles manually annotated for evaluation purposes.
  arXiv  Detail & Related papers  (2024-06-20T22:03:21Z)
• SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature [80.49349719239584]
  We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following demonstrations for 54 tasks. SciRIFF is the first dataset focused on extracting and synthesizing information from research literature across a wide range of scientific fields.
  arXiv  Detail & Related papers  (2024-06-10T21:22:08Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models [57.96527452844273]
  We introduce SciInstruct, a suite of scientific instructions for training scientific language models capable of college-level scientific reasoning. We curated a diverse and high-quality dataset encompassing physics, chemistry, math, and formal proofs. To verify the effectiveness of SciInstruct, we fine-tuned different language models with SciInstruct, i.e., ChatGLM3 (6B and 32B), Llama3-8B-Instruct, and Mistral-7B: MetaMath.
  arXiv  Detail & Related papers  (2024-01-15T20:22:21Z)
• SciFact-Open: Towards open-domain scientific claim verification [61.288725621156864]
  We present SciFact-Open, a new test collection designed to evaluate the performance of scientific claim verification systems. We collect evidence for scientific claims by pooling and annotating the top predictions of four state-of-the-art scientific claim verification models. We find that systems developed on smaller corpora struggle to generalize to SciFact-Open, exhibiting performance drops of at least 15 F1.
  arXiv  Detail & Related papers  (2022-10-25T05:45:00Z)
• A Hybrid Science-Guided Machine Learning Approach for Modeling and Optimizing Chemical Processes [0.0]
  Hybrid process modeling and optimization is combined with a science-guided machine learning (SGML) approach. For applying ML to improve models, we present expositions of the sub-categories of direct serial and parallel hybrid modeling. For applying scientific principles to improve ML models, we discuss the sub-categories of science-guided design, learning and refinement.
  arXiv  Detail & Related papers  (2021-12-02T18:24:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.