SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines

• URL: http://arxiv.org/abs/2509.21320v2
• Date: Wed, 29 Oct 2025 16:14:05 GMT
• Title: SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines
• Authors: Yizhou Wang, Chen Tang, Han Deng, Jiabei Xiao, Jiaqi Liu, Jianyu Wu, Jun Yao, Pengze Li, Encheng Su, Lintao Wang, Guohang Zhuang, Yuchen Ren, Ben Fei, Ming Hu, Xin Chen, Dongzhan Zhou, Junjun He, Xiangyu Yue, Zhenfei Yin, Jiamin Wu, Qihao Zheng, Yuhao Zhou, Huihui Xu, Chenglong Ma, Yan Lu, Wenlong Zhang, Chunfeng Song, Philip Torr, Shixiang Tang, Xinzhu Ma, Wanli Ouyang, Lei Bai,
• Abstract summary: We present a scientific reasoning foundation model that aligns natural language with heterogeneous scientific representations.<n>The model is pretrained on a 206B-token corpus spanning scientific text, pure sequences, and sequence-text pairs, then aligned via SFT on 40M instructions.<n>It supports four capability families, covering up to 103 tasks across: (i) faithful translation between text and scientific formats, (ii) text/knowledge extraction, (iii) property prediction, (iv) property classification, (v) unconditional and conditional sequence generation and design.
• Score: 112.78540935201558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a scientific reasoning foundation model that aligns natural language with heterogeneous scientific representations. The model is pretrained on a 206B-token corpus spanning scientific text, pure sequences, and sequence-text pairs, then aligned via SFT on 40M instructions, annealed cold-start bootstrapping to elicit long-form chain-of-thought, and reinforcement learning with task-specific reward shaping, which instills deliberate scientific reasoning. It supports four capability families, covering up to 103 tasks across workflows: (i) faithful translation between text and scientific formats, (ii) text/knowledge extraction, (iii) property prediction, (iv) property classification, (v) unconditional and conditional sequence generation and design. Compared with specialist systems, our approach broadens instruction coverage, improves cross-domain generalization, and enhances fidelity. We detail data curation and training and show that cross-discipline learning strengthens transfer and downstream reliability. The model, instruct tuning datasets and the evaluation code are open-sourced at https://huggingface.co/SciReason and https://github.com/open-sciencelab/SciReason.

Related papers

• Data Darwinism Part I: Unlocking the Value of Scientific Data for Pre-training [39.148751989348774]
  We introduce Data Darwinism, a ten-level taxonomy that conceptualizes data-model co-evolution.<n>We validate this on scientific literature by constructing Darwin-Science, a 900B-token corpus.<n>We release the Darwin-Science corpus and daci-origin models to enable principled, co-evolutionary development.
  arXiv  Detail & Related papers  (2026-02-08T05:06:34Z)
• Lost in Tokenization: Context as the Key to Unlocking Biomolecular Understanding in Scientific LLMs [78.18336140706471]
  Sci-LLMs have emerged as a promising frontier for accelerating biological discovery.<n>Current strategies limit Sci-LLMs' reasoning capacity when processing raw biomolecular sequences.<n>We show that a more effective strategy is to provide Sci-LLMs with high-level structured context.
  arXiv  Detail & Related papers  (2025-10-27T09:03:21Z)
• CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning [63.44477226386808]
  Chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks.<n>But it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning.<n>We propose CoT-Evo, an evolutionary CoT distillation framework to overcome this problem.
  arXiv  Detail & Related papers  (2025-10-15T05:29:56Z)
• SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature [80.49349719239584]
  We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following demonstrations for 54 tasks. SciRIFF is the first dataset focused on extracting and synthesizing information from research literature across a wide range of scientific fields.
  arXiv  Detail & Related papers  (2024-06-10T21:22:08Z)
• Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables [22.18384189336634]
  HeterFC is a word-level Heterogeneous-graph-based model for Fact Checking over unstructured and structured information. We perform information propagation via a relational graph neural network, interactions between claims and evidence. We introduce a multitask loss function to account for potential inaccuracies in evidence retrieval.
  arXiv  Detail & Related papers  (2024-02-20T14:10:40Z)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models [57.96527452844273]
  We introduce SciInstruct, a suite of scientific instructions for training scientific language models capable of college-level scientific reasoning. We curated a diverse and high-quality dataset encompassing physics, chemistry, math, and formal proofs. To verify the effectiveness of SciInstruct, we fine-tuned different language models with SciInstruct, i.e., ChatGLM3 (6B and 32B), Llama3-8B-Instruct, and Mistral-7B: MetaMath.
  arXiv  Detail & Related papers  (2024-01-15T20:22:21Z)
• ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Science [0.0]
  Large language models record impressive performance on many natural language processing tasks. Retrieval augmentation offers an effective solution by retrieving context from external knowledge sources. We propose a novel structure-aware retrieval augmented language model that accommodates document structure during retrieval augmentation.
  arXiv  Detail & Related papers  (2023-11-21T02:02:46Z)
• CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding [23.930041685595775]
  We present an in-depth study of cite-worthiness detection in English, where a sentence is labelled for whether or not it cites an external source. CiteWorth is high-quality, challenging, and suitable for studying problems such as domain adaptation.
  arXiv  Detail & Related papers  (2021-05-23T11:08:45Z)
• Improving Text Generation with Student-Forcing Optimal Transport [122.11881937642401]
  We propose using optimal transport (OT) to match the sequences generated in training and testing modes. An extension is also proposed to improve the OT learning, based on the structural and contextual information of the text sequences. The effectiveness of the proposed method is validated on machine translation, text summarization, and text generation tasks.
  arXiv  Detail & Related papers  (2020-10-12T19:42:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.