Monkey King Bang: A Unified Scientific Multimodal Foundation Model

• URL: http://arxiv.org/abs/2607.20557v1
• Date: Fri, 17 Jul 2026 13:56:50 GMT
• Title: Monkey King Bang: A Unified Scientific Multimodal Foundation Model
• Abstract summary: We introduce MKB, a unified scientific multimodal model for both understanding and generation.<n> MKB covers six scientific branches, including DNA, RNA, proteins, small molecules, earth science, and medical images.<n>It supports native outputs such as biological sequences, molecular strings, meteorological fields, and segmentation masks.
• Score: 26.409792993729948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific discovery is increasingly shifting from isolated disciplines to multi-domain reasoning, and AI for science faces a similar transition. Existing systems are either specialised for individual domains or unify scientific data mainly through text tokenisation and prompt-based interfaces, limiting their ability to handle diverse scientific inputs, produce modality-native outputs, and support joint understanding, reasoning, and generation across scientific domains. We introduce MKB, a unified scientific multimodal model for both understanding and generation, built around a shared Transformer backbone and modality-tailored encoders, adapters, and decoders. MKB covers six scientific branches, including DNA, RNA, proteins, small molecules, earth science, and medical images, and supports native outputs such as biological sequences, molecular strings, meteorological fields, and segmentation masks. Training follows a two-stage modality-then-language curriculum: Stage 1 aligns modality-specific components with the frozen backbone, and Stage 2 consolidates them with the language backbone using mixed scientific and general corpora. Experiments show that MKB achieves competitive scientific understanding across biological and molecular benchmarks, produces high-fidelity native outputs for weather forecasting, biological generation, and medical-image segmentation, and largely retains the general capabilities of its Qwen3-VL backbone. These results demonstrate the feasibility of the proposed paradigm, suggesting that shared-backbone models with modality-tailored components can provide a promising foundation for future cross-domain scientific multimodal exploration. The model and code are publicly available at https://github.com/Shanghai-Academy-of-AI-For-Science/MKB and https://huggingface.co/sais-org/MKB.

Related papers

• S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation [31.901449470705618]
  We present S1- Omni, a unified multimodal reasoning model for scientific understanding, prediction, and generation.<n>The architecture of S1- Omni is built upon three core components: unified representation of scientific data, natural-world knowledge alignment, and decoding for domain-specific tasks.<n>S1- Omni is trained on S1- Omni-Corpus, which covers 200 scientific tasks and contains millions of reasoning samples, and is evaluated on over 60 scientific benchmarks.
  arXiv  Detail & Related papers  (2026-07-17T06:56:08Z)
• Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences [89.71422932447423]
  LOGOS is a scientific generative language model that unifies heterogeneous tasks across the natural sciences.<n>By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner.<n>Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines.
  arXiv  Detail & Related papers  (2026-06-15T16:14:53Z)
• SciCore-Mol: Augmenting Large Language Models with Pluggable Molecular Cognition Modules [37.73750078629302]
  SciCore-Mol is a modular framework that bridges the gap between discrete linguistic symbols and topological molecular or continuous reaction data.<n>Our experiments on diverse chemical tasks demonstrate that SciCore-Mol achieves strong comprehensive performance across molecular understanding, generation, reaction prediction, and general chemistry knowledge.<n>This work provides a systematic blueprint for equipping Large Language Models with scientific expertise through decoupled, pluggable, and flexibly orchestrated modules.
  arXiv  Detail & Related papers  (2026-05-21T10:37:53Z)
• A unified multimodal understanding and generation model for cross-disciplinary scientific research [20.510417042775973]
  FuXi-Uni is a native unified multimodal model for scientific understanding and high-fidelity generation across scientific domains within a single architecture.<n>Specifically, FuXi-Uni aligns cross-disciplinary scientific tokens within natural language tokens and employs science decoder to reconstruct scientific tokens.<n>In Earth system modeling, the model supports global weather forecasting, tropical cyclone (TC) forecast editing, and spatial downscaling driven by only language instructions.
  arXiv  Detail & Related papers  (2026-01-04T04:31:38Z)
• SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence [99.30934038146965]
  SciEvalKit focuses on the core competencies of scientific intelligence.<n>It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science.<n>The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.
  arXiv  Detail & Related papers  (2025-12-26T17:36:02Z)
• Learning Cell-Aware Hierarchical Multi-Modal Representations for Robust Molecular Modeling [74.25438319700929]
  We propose CHMR (Cell-aware Hierarchical Multi-modal Representations), a robust framework that models local-global dependencies between molecules and cellular responses.<n> evaluated on nine public benchmarks spanning 728 tasks, CHMR outperforms state-of-the-art baselines.<n>Results demonstrate the advantage of hierarchy-aware, multimodal learning for reliable and biologically grounded molecular representations.
  arXiv  Detail & Related papers  (2025-11-26T07:15:00Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Nature Language Model: Deciphering the Language of Nature for Scientific Discovery [105.55751854768297]
  Foundation models have revolutionized natural language processing and artificial intelligence.<n>We introduce Nature Language Model (NatureLM), a sequence-based science foundation model for scientific discovery.
  arXiv  Detail & Related papers  (2025-02-11T13:08:03Z)
• MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language [0.4631438140637248]
  MAMMAL is a versatile method applied to create a multi-task foundation model that learns from large-scale biological datasets across diverse modalities.<n> evaluated on eleven diverse downstream tasks, it reaches a new state of the art (SOTA) in nine tasks and is comparable to SOTA in two tasks.<n> explored Alphafold 3 binding prediction capabilities on antibody-antigen and nanobody-antigen complexes showing significantly better classification performance of MAMMAL in 3 out of 4 targets.
  arXiv  Detail & Related papers  (2024-10-28T20:45:52Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.