ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-tuned Large Language Model in Token Granularity

• URL: http://arxiv.org/abs/2411.12000v2
• Date: Fri, 06 Dec 2024 21:08:10 GMT
• Title: ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-tuned Large Language Model in Token Granularity
• Abstract summary: ByteScience is a non-profit cloud-based auto fine-tuned Large Language Model (LLM) platform.<n>It is designed to extract structured scientific data and synthesize new scientific knowledge from vast scientific corpora.<n>The platform achieves remarkable accuracy with only a small amount of well-annotated articles.
• Score: 13.978222668670192
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural Language Processing (NLP) is widely used to supply summarization ability from long context to structured information. However, extracting structured knowledge from scientific text by NLP models remains a challenge because of its domain-specific nature to complex data preprocessing and the granularity of multi-layered device-level information. To address this, we introduce ByteScience, a non-profit cloud-based auto fine-tuned Large Language Model (LLM) platform, which is designed to extract structured scientific data and synthesize new scientific knowledge from vast scientific corpora. The platform capitalizes on DARWIN, an open-source, fine-tuned LLM dedicated to natural science. The platform was built on Amazon Web Services (AWS) and provides an automated, user-friendly workflow for custom model development and data extraction. The platform achieves remarkable accuracy with only a small amount of well-annotated articles. This innovative tool streamlines the transition from the science literature to structured knowledge and data and benefits the advancements in natural informatics.

Related papers

• Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models [2.2508510871478014]
  eolas is a pipeline that transforms scientific documents into knowledge graphs aligned with a specified ontology.<n>We demonstrate eolas effectiveness in extracting useful information for scientists studying materials designed to endure the extreme temperatures and radiation levels found in fusion reactors.
  arXiv  Detail & Related papers  (2026-09-15T15:05:57Z)
• Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences [89.71422932447423]
  LOGOS is a scientific generative language model that unifies heterogeneous tasks across the natural sciences.<n>By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner.<n>Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines.
  arXiv  Detail & Related papers  (2026-06-15T16:14:53Z)
• On the Emergence and Test-Time Use of Structural Information in Large Language Models [52.28603345019514]
  We study how language models learn abstract structures and utilize the learnt structural information at test-time.<n>We empirically show that the emergence of learning structural information correlates with complex reasoning tasks.
  arXiv  Detail & Related papers  (2026-01-25T15:02:25Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Generative Hierarchical Materials Search [91.93125016916463]
  We propose Generative Hierarchical Materials Search (GenMS) for controllable generation of crystal structures. GenMS consists of (1) a language model that takes high-level natural language as input and generates intermediate textual information about a crystal. GenMS additionally uses a graph neural network to predict properties (e.g., formation energy) from the generated crystal structures.
  arXiv  Detail & Related papers  (2024-09-10T17:51:28Z)
• Knowledge AI: Fine-tuning NLP Models for Facilitating Scientific Knowledge Extraction and Understanding [0.0]
  This project investigates the efficacy of Large Language Models (LLMs) in understanding and extracting scientific knowledge across specific domains. We employ pre-trained models and fine-tune them on datasets in the scientific domain.
  arXiv  Detail & Related papers  (2024-08-04T01:32:09Z)
• Human-artificial intelligence teaming for scientific information extraction from data-driven additive manufacturing research using large language models [3.0061386772253784]
  Data-driven research in Additive Manufacturing (AM) has gained significant success in recent years. This has led to a plethora of scientific literature to emerge. It requires substantial effort and time to extract scientific information from these works. We propose a framework that enables collaboration between AM and AI experts to continuously extract scientific information from data-driven AM literature.
  arXiv  Detail & Related papers  (2024-07-26T15:43:52Z)
• From Text to Insight: Large Language Models for Materials Science Data Extraction [4.08853418443192]
  The vast majority of materials science knowledge exists in unstructured natural language. Structured data is crucial for innovative and systematic materials design. The advent of large language models (LLMs) represents a significant shift.
  arXiv  Detail & Related papers  (2024-07-23T22:23:47Z)
• SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature [80.49349719239584]
  We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following demonstrations for 54 tasks. SciRIFF is the first dataset focused on extracting and synthesizing information from research literature across a wide range of scientific fields.
  arXiv  Detail & Related papers  (2024-06-10T21:22:08Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• Agent-based Learning of Materials Datasets from Scientific Literature [0.0]
  We develop a chemist AI agent, powered by large language models (LLMs), to create structured datasets from natural language text. Our chemist AI agent, Eunomia, can plan and execute actions by leveraging the existing knowledge from decades of scientific research articles.
  arXiv  Detail & Related papers  (2023-12-18T20:29:58Z)
• Large Language Models for Scientific Synthesis, Inference and Explanation [56.41963802804953]
  We show how large language models can perform scientific synthesis, inference, and explanation. We show that the large language model can augment this "knowledge" by synthesizing from the scientific literature. This approach has the further advantage that the large language model can explain the machine learning system's predictions.
  arXiv  Detail & Related papers  (2023-10-12T02:17:59Z)
• Accelerated materials language processing enabled by GPT [5.518792725397679]
  We develop generative transformer (GPT)-enabled pipelines for materials language processing. First, we develop a GPT-enabled document classification method for screening relevant documents. Secondly, for NER task, we design an entity-centric prompts, and learning few-shot of them improved the performance. Finally, we develop an GPT-enabled extractive QA model, which provides improved performance and shows the possibility of automatically correcting annotations.
  arXiv  Detail & Related papers  (2023-08-18T07:31:13Z)
• The Semantic Scholar Open Data Platform [79.4493235243312]
  Semantic Scholar (S2) is an open data platform and website aimed at accelerating science by helping scholars discover and understand scientific literature. We combine public and proprietary data sources using state-of-the-art techniques for scholarly PDF content extraction and automatic knowledge graph construction. The graph includes advanced semantic features such as structurally parsed text, natural language summaries, and vector embeddings.
  arXiv  Detail & Related papers  (2023-01-24T17:13:08Z)
• Structured information extraction from complex scientific text with fine-tuned large language models [55.96705756327738]
  We present a simple sequence-to-sequence approach to joint named entity recognition and relation extraction. The approach leverages a pre-trained large language model (LLM), GPT-3, that is fine-tuned on approximately 500 pairs of prompts. This approach represents a simple, accessible, and highly-flexible route to obtaining large databases of structured knowledge extracted from unstructured text.
  arXiv  Detail & Related papers  (2022-12-10T07:51:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.