Exploring LLMs for Scientific Information Extraction Using The SciEx Framework

• URL: http://arxiv.org/abs/2512.10004v1
• Date: Wed, 10 Dec 2025 19:00:20 GMT
• Title: Exploring LLMs for Scientific Information Extraction Using The SciEx Framework
• Authors: Sha Li, Ayush Sadekar, Nathan Self, Yiqi Su, Lars Andersland, Mira Chaplin, Annabel Zhang, Hyoju Yang, James B Henderson, Krista Wigginton, Linsey Marr, T. M. Murali, Naren Ramakrishnan,
• Abstract summary: Large language models (LLMs) are touted as powerful tools for automating scientific information extraction.<n>We present SciEx, a modular and composable framework that decouples key components including PDF parsing, multi-modal retrieval, extraction, and aggregation.<n>We evaluate SciEx on datasets spanning three scientific topics for its ability to extract fine-grained information accurately and consistently.
• Score: 12.534492015126757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly touted as powerful tools for automating scientific information extraction. However, existing methods and tools often struggle with the realities of scientific literature: long-context documents, multi-modal content, and reconciling varied and inconsistent fine-grained information across multiple publications into standardized formats. These challenges are further compounded when the desired data schema or extraction ontology changes rapidly, making it difficult to re-architect or fine-tune existing systems. We present SciEx, a modular and composable framework that decouples key components including PDF parsing, multi-modal retrieval, extraction, and aggregation. This design streamlines on-demand data extraction while enabling extensibility and flexible integration of new models, prompting strategies, and reasoning mechanisms. We evaluate SciEx on datasets spanning three scientific topics for its ability to extract fine-grained information accurately and consistently. Our findings provide practical insights into both the strengths and limitations of current LLM-based pipelines.

Related papers

• Large language models in materials science and the need for open-source approaches [3.35950184561189]
  Review examines recent large language models (LLMs) applications across the materials discovery pipeline.<n>We highlight how LLMs extract valuable information such as synthesis conditions from text.<n> benchmark results demonstrate that open-source alternatives can match performance while offering greater transparency, cost-effectiveness, and data privacy.<n>As open-source models continue to improve, we advocate their broader adoption to build accessible, flexible, and community-driven AI platforms for scientific discovery.
  arXiv  Detail & Related papers  (2025-11-10T00:05:20Z)
• LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature [60.879220305044726]
  We propose a multi-modal toolbox that employs large language models (LLMs) and vision language models (VLMs) to automatically extract and organize synthesis procedures and performance data.<n>We curated 81k open-access papers, yielding LeMat- Synth (v 1.0): a dataset containing synthesis procedures spanning 35 synthesis methods and 16 material classes.<n>We release a modular, open-source library designed to support community-driven extension to new corpora and synthesis domains.
  arXiv  Detail & Related papers  (2025-10-28T17:58:18Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• Who Gets Cited Most? Benchmarking Long-Context Language Models on Scientific Articles [81.89404347890662]
  SciTrek is a novel question-answering benchmark designed to evaluate the long-context reasoning capabilities of large language models (LLMs) using scientific articles.<n>Our analysis reveals systematic shortcomings in models' abilities to perform basic numerical operations and accurately locate specific information in long contexts.
  arXiv  Detail & Related papers  (2025-09-25T11:36:09Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Context-Aware Scientific Knowledge Extraction on Linked Open Data using Large Language Models [0.0]
  This paper introduces WISE (Workflow for Intelligent Scientific Knowledge Extraction), a system to extract, refine, and rank query-specific knowledge.<n>WISE delivers detailed, organized answers by systematically exploring and synthesizing knowledge from diverse sources.
  arXiv  Detail & Related papers  (2025-06-21T04:22:34Z)
• MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs [48.73595915402094]
  MOLE is a framework that automatically extracts metadata attributes from scientific papers covering datasets of languages other than Arabic.<n>Our methodology processes entire documents across multiple input formats and incorporates robust validation mechanisms for consistent output.
  arXiv  Detail & Related papers  (2025-05-26T10:31:26Z)
• Collage: Decomposable Rapid Prototyping for Information Extraction on Scientific PDFs [15.610004991273005]
  We present Collage, a tool designed for rapid prototyping, visualization, and evaluation of different information extraction models on scientific PDFs.<n>We enable both developers and users of NLP-based tools to inspect, debug, and better understand modeling pipelines by providing granular views of intermediate states of processing.
  arXiv  Detail & Related papers  (2024-10-30T22:00:34Z)
• SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents [49.54155332262579]
  We release a new entity and relation extraction dataset for entities related to datasets, methods, and tasks in scientific articles. Our dataset contains 106 manually annotated full-text scientific publications with over 24k entities and 12k relations.
  arXiv  Detail & Related papers  (2024-10-28T15:56:49Z)
• MatViX: Multimodal Information Extraction from Visually Rich Articles [6.349779979863784]
  In materials science, extracting structured information from research articles can accelerate the discovery of new materials. We introduce textscMatViX, a benchmark consisting of $324$ full-length research articles and $1,688$ complex structured files. These files are extracted from text, tables, and figures in full-length documents, providing a comprehensive challenge for MIE.
  arXiv  Detail & Related papers  (2024-10-27T16:13:58Z)
• From Text to Insight: Large Language Models for Materials Science Data Extraction [4.08853418443192]
  The vast majority of materials science knowledge exists in unstructured natural language.<n>Structured data is crucial for innovative and systematic materials design.<n>The advent of large language models (LLMs) represents a significant shift.
  arXiv  Detail & Related papers  (2024-07-23T22:23:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.