PolyReal: A Benchmark for Real-World Polymer Science Workflows

• URL: http://arxiv.org/abs/2604.02934v1
• Date: Fri, 03 Apr 2026 10:05:30 GMT
• Title: PolyReal: A Benchmark for Real-World Polymer Science Workflows
• Abstract summary: PolyReal is a novel benchmark grounded in real-world scientific practices.<n>It covers five critical capabilities: (1) knowledge application; (2) lab safety analysis; (3) experiment mechanism reasoning; (4) raw data extraction; and (5) performance & application exploration.
• Score: 76.8670713411835
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) excel in general domains but struggle with complex, real-world science. We posit that polymer science, an interdisciplinary field spanning chemistry, physics, biology, and engineering, is an ideal high-stakes testbed due to its diverse multimodal data. Yet, existing benchmarks related to polymer science largely overlook real-world workflows, limiting their practical utility and failing to systematically evaluate MLLMs across the full, practice-grounded lifecycle of experimentation. We introduce PolyReal, a novel multimodal benchmark grounded in real-world scientific practices to evaluate MLLMs on the full lifecycle of polymer experimentation. It covers five critical capabilities: (1) foundational knowledge application; (2) lab safety analysis; (3) experiment mechanism reasoning; (4) raw data extraction; and (5) performance & application exploration. Our evaluation of leading MLLMs on PolyReal reveals a capability imbalance. While models perform well on knowledge-intensive reasoning (e.g., Experiment Mechanism Reasoning), they drop sharply on practice-based tasks (e.g., Lab Safety Analysis and Raw Data Extraction). This exposes a severe gap between abstract scientific knowledge and its practical, context-dependent application, showing that these real-world tasks remain challenging for MLLMs. Thus, PolyReal helps address this evaluation gap and provides a practical benchmark for assessing AI systems in real-world scientific workflows.

Related papers

• SciMIF: Understanding Multimodal Instruction Following in Scientific Domains [50.54744513908656]
  We introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions.<n>Based on an analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy.<n>We develop a high-fidelity instruction injection pipeline to systematically augment existing scientific datasets.
  arXiv  Detail & Related papers  (2026-08-26T16:30:20Z)
• Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows [203.3527268311731]
  We present an operational SGI definition grounded in the Practical Inquiry Model (PIM)<n>We operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning.<n>Our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
  arXiv  Detail & Related papers  (2025-12-18T12:44:36Z)
• ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows [82.07367406991678]
  Large Language Models (LLMs) have extended their impact beyond Natural Language Processing.<n>Among these, computer-using agents are capable of interacting with operating systems as humans do.<n>We introduce ScienceBoard, which encompasses a realistic, multi-domain environment featuring dynamic and visually rich scientific software.
  arXiv  Detail & Related papers  (2025-05-26T12:27:27Z)
• CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning [12.396302011805755]
  We introduce CURIE, a benchmark to measure the potential of Large Language Models (LLMs) in scientific problem-solving.<n>This benchmark introduces ten challenging tasks with a total of 580 problems and solution pairs curated by experts in six disciplines.<n>We evaluate a range of closed and open LLMs on tasks in CURIE which requires domain expertise, comprehension of long in-context information,and multi-step reasoning.
  arXiv  Detail & Related papers  (2025-03-14T17:53:03Z)
• Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning [51.11965014462375]
  Multimodal Large Language Models (MLLMs) integrate text, images, and other modalities.<n>This paper argues that MLLMs can significantly advance scientific reasoning across disciplines such as mathematics, physics, chemistry, and biology.
  arXiv  Detail & Related papers  (2025-02-05T04:05:27Z)
• A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery [68.48094108571432]
  Large language models (LLMs) have revolutionized the way text and other modalities of data are handled. We aim to provide a more holistic view of the research landscape by unveiling cross-field and cross-modal connections between scientific LLMs.
  arXiv  Detail & Related papers  (2024-06-16T08:03:24Z)
• Workflow Provenance in the Lifecycle of Scientific Machine Learning [1.6118907823528272]
  We leverage workflow techniques to build a holistic view to support the lifecycle of scientific ML. We contribute with (i) characterization of the lifecycle and taxonomy for data analyses; (ii) design principles to build this view, with a W3C PROV compliant data representation and a reference system architecture; and (iii) lessons learned after an evaluation in an Oil & Gas case using an HPC cluster with 393 nodes and 946 GPUs.
  arXiv  Detail & Related papers  (2020-09-30T13:09:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.