Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

• URL: http://arxiv.org/abs/2606.16905v1
• Date: Mon, 15 Jun 2026 16:14:53 GMT
• Title: Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences
• Abstract summary: LOGOS is a scientific generative language model that unifies heterogeneous tasks across the natural sciences.<n>By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner.<n>Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines.
• Score: 89.71422932447423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a single autoregressive framework based on a shared scientific grammar. It encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of "one model fits all" in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future of AI for Science (AI4S) may not lie in building an independent technical stack that is separated from large language models (LLMs). Instead, it may depend on deeply aligning scientific foundation models with LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.

Related papers

• Unsupervised Learning of Inter-Object Relationships via Group Homomorphism [3.8731899020772165]
  In this study, we propose an unsupervised representation learning method based on a hierarchical relationship in group operations.<n>We experimentally demonstrate that the model can segment multiple objects into individual slots without any ground-truth labels.<n>Results suggest that by introducing algebraic geometric constraints, physically interpretable "disentangled representations" can be acquired.
  arXiv  Detail & Related papers  (2026-04-22T07:17:00Z)
• Hyperbolic Large Language Models [7.483401973996036]
  Large language models (LLMs) have achieved remarkable success and demonstrated superior performance across various tasks.<n>However, many real-world data exhibit highly non-Euclidean latent hierarchical anatomy, such as protein networks, transportation networks, financial networks, brain networks, and linguistic structures or syntactic trees in natural languages.<n>We provide a comprehensive and contextual exposition of recent advancements in LLMs that leverage hyperbolic geometry as a representation space to enhance semantic representation learning and multi-scale reasoning.
  arXiv  Detail & Related papers  (2025-09-06T15:56:46Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures [49.19753720526998]
  We derive theoretical scaling laws for neural network performance on synthetic datasets.<n>We validate that convolutional networks, whose structure aligns with that of the generative process through locality and weight sharing, enjoy a faster scaling of performance.<n>This finding clarifies the architectural biases underlying neural scaling laws and highlights how representation learning is shaped by the interaction between model architecture and the statistical properties of data.
  arXiv  Detail & Related papers  (2025-05-11T17:44:14Z)
• Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning [29.745218855471787]
  Tokenization is a necessary component within the current architecture of many language mod-els.<n>We argue that tokenization is necessary for reasonably human-like language performance.<n>We discuss implications for architectural choices, meaning construction, the primacy of language for thought.
  arXiv  Detail & Related papers  (2024-12-14T18:18:52Z)
• Training Neural Networks as Recognizers of Formal Languages [87.06906286950438]
  We train and evaluate neural networks directly as binary classifiers of strings.<n>We provide results on a variety of languages across the Chomsky hierarchy for three neural architectures.<n>Our contributions will facilitate theoretically sound empirical testing of language recognition claims in future work.
  arXiv  Detail & Related papers  (2024-11-11T16:33:25Z)
• In-Context Language Learning: Architectures and Algorithms [73.93205821154605]
  We study ICL through the lens of a new family of model problems we term in context language learning (ICLL) We evaluate a diverse set of neural sequence models on regular ICLL tasks.
  arXiv  Detail & Related papers  (2024-01-23T18:59:21Z)
• Rethinking Relational Encoding in Language Model: Pre-Training for General Sequences [23.806325599416134]
  Language model pre-training fails at modeling per-sequence relations in non-natural language domains. We develop a framework that couples LMPT with deep structure-preserving metric learning to produce richer embeddings. Our approach offers notable performance improvements on downstream tasks.
  arXiv  Detail & Related papers  (2021-03-18T15:51:04Z)
• Reprogramming Language Models for Molecular Representation Learning [65.00999660425731]
  We propose Representation Reprogramming via Dictionary Learning (R2DL) for adversarially reprogramming pretrained language models for molecular learning tasks. The adversarial program learns a linear transformation between a dense source model input space (language data) and a sparse target model input space (e.g., chemical and biological molecule data) using a k-SVD solver. R2DL achieves the baseline established by state of the art toxicity prediction models trained on domain-specific data and outperforms the baseline in a limited training-data setting.
  arXiv  Detail & Related papers  (2020-12-07T05:50:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.