SciGPT: A Large Language Model for Scientific Literature Understanding and Knowledge Discovery

• URL: http://arxiv.org/abs/2509.08032v1
• Date: Tue, 09 Sep 2025 16:09:19 GMT
• Title: SciGPT: A Large Language Model for Scientific Literature Understanding and Knowledge Discovery
• Authors: Fengyu She, Nan Wang, Hongfei Wu, Ziyi Wan, Jingmian Wang, Chang Wang,
• Abstract summary: This paper presents SciGPT, a domain-adapted model for scientific literature understanding and ScienceBench, an open source benchmark tailored to evaluate scientific LLMs.<n>Built on the Qwen3 architecture, SciGPT incorporates three key innovations: (1) low-cost domain distillation via a two-stage pipeline to balance performance and efficiency; (2) a Sparse Mixture-of-Experts attention mechanism that cuts memory consumption by 55% for 32,000 long-token reasoning; and (3) knowledge-aware adaptation integrating domain-specific nuances.<n> Experimental results on ScienceBench show that SciGPT outperforms GPT-4o in core scientific tasks including sequence
• Score: 3.779883844533933
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific literature is growing exponentially, creating a critical bottleneck for researchers to efficiently synthesize knowledge. While general-purpose Large Language Models (LLMs) show potential in text processing, they often fail to capture scientific domain-specific nuances (e.g., technical jargon, methodological rigor) and struggle with complex scientific tasks, limiting their utility for interdisciplinary research. To address these gaps, this paper presents SciGPT, a domain-adapted foundation model for scientific literature understanding and ScienceBench, an open source benchmark tailored to evaluate scientific LLMs. Built on the Qwen3 architecture, SciGPT incorporates three key innovations: (1) low-cost domain distillation via a two-stage pipeline to balance performance and efficiency; (2) a Sparse Mixture-of-Experts (SMoE) attention mechanism that cuts memory consumption by 55\% for 32,000-token long-document reasoning; and (3) knowledge-aware adaptation integrating domain ontologies to bridge interdisciplinary knowledge gaps. Experimental results on ScienceBench show that SciGPT outperforms GPT-4o in core scientific tasks including sequence labeling, generation, and inference. It also exhibits strong robustness in unseen scientific tasks, validating its potential to facilitate AI-augmented scientific discovery.

Related papers

• SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature [52.36039386997026]
  We introduce SciRAG, an open-source framework for scientific literature exploration.<n>We introduce three key innovations: (1) adaptive retrieval that flexibly alternates between sequential and parallel evidence gathering; (2) citation-aware symbolic reasoning that leverages citation graphs to organize and filter documents; and (3) outline-guided synthesis that plans, critiques, and refines answers to ensure coherence and transparent attribution.
  arXiv  Detail & Related papers  (2025-11-18T11:09:19Z)
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning [53.82037883518254]
  We introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks.<n>We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks.
  arXiv  Detail & Related papers  (2025-08-26T17:04:23Z)
• Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning [59.518397361341556]
  We present the Scientists' First Exam (SFE) benchmark, designed to evaluate the scientific cognitive capacities of Multimodal Large Language Models (MLLMs)<n>SFE comprises 830 expert-verified VQA pairs across three question types, spanning 66 multimodal tasks across five high-value disciplines.<n>Experiments reveal that current state-of-the-art GPT-o3 and InternVL-3 achieve only 34.08% and 26.52% on SFE, highlighting significant room for MLLMs to improve in scientific realms.
  arXiv  Detail & Related papers  (2025-06-12T09:29:16Z)
• Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery [35.888956949646]
  Large Language Models (LLMs) are transforming scientific research by reshaping the scientific method.<n>LLMs are involved in experimental design, data analysis, and enhancing productivity, particularly in chemistry and biology.<n>Transition to AI-driven science raises ethical questions about creativity, oversight, and responsibility.
  arXiv  Detail & Related papers  (2025-05-22T10:05:48Z)
• HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights [72.82973609312178]
  HiPerRAG is a workflow to index and retrieve knowledge from more than 3.6 million scientific articles.<n>At its core are Oreo, a high- throughput model for multimodal document parsing, and ColTrast, a query-aware encoder fine-tuning algorithm.<n>HiPerRAG delivers robust performance on existing scientific question answering benchmarks and two new benchmarks introduced in this work.
  arXiv  Detail & Related papers  (2025-05-07T22:50:23Z)
• OmniScience: A Domain-Specialized LLM for Scientific Reasoning and Discovery [12.306721865990053]
  We introduce OmniScience, a specialized large reasoning model for general science.<n>We develop a battery agent that efficiently ranks molecules as potential electrolyte solvents or additives.<n>We demonstrate via ablation experiments that domain adaptive pretraining and reasoning-based knowledge distillation are critical to attain our performance levels.
  arXiv  Detail & Related papers  (2025-03-22T01:18:59Z)
• Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation [58.064940977804596]
  A plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently.<n>Ethical concerns regarding shortcomings of these tools and potential for misuse take a particularly prominent place in our discussion.
  arXiv  Detail & Related papers  (2025-02-07T18:26:45Z)
• SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding [22.131371019641417]
  Despite Large Language Models' success, they face challenges in scientific literature understanding.<n>We propose a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT)<n>We present a suite of LLMs: SciLitLLM, specialized in scientific literature understanding.
  arXiv  Detail & Related papers  (2024-08-28T05:41:52Z)
• Scientific Large Language Models: A Survey on Biological & Chemical Domains [47.97810890521825]
  Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration.
  arXiv  Detail & Related papers  (2024-01-26T05:33:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.