A unified multimodal understanding and generation model for cross-disciplinary scientific research

• URL: http://arxiv.org/abs/2601.01363v1
• Date: Sun, 04 Jan 2026 04:31:38 GMT
• Title: A unified multimodal understanding and generation model for cross-disciplinary scientific research
• Abstract summary: FuXi-Uni is a native unified multimodal model for scientific understanding and high-fidelity generation across scientific domains within a single architecture.<n>Specifically, FuXi-Uni aligns cross-disciplinary scientific tokens within natural language tokens and employs science decoder to reconstruct scientific tokens.<n>In Earth system modeling, the model supports global weather forecasting, tropical cyclone (TC) forecast editing, and spatial downscaling driven by only language instructions.
• Score: 20.510417042775973
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Scientific discovery increasingly relies on integrating heterogeneous, high-dimensional data across disciplines nowadays. While AI models have achieved notable success across various scientific domains, they typically remain domain-specific or lack the capability of simultaneously understanding and generating multimodal scientific data, particularly for high-dimensional data. Yet, many pressing global challenges and scientific problems are inherently cross-disciplinary and require coordinated progress across multiple fields. Here, we present FuXi-Uni, a native unified multimodal model for scientific understanding and high-fidelity generation across scientific domains within a single architecture. Specifically, FuXi-Uni aligns cross-disciplinary scientific tokens within natural language tokens and employs science decoder to reconstruct scientific tokens, thereby supporting both natural language conversation and scientific numerical prediction. Empirically, we validate FuXi-Uni in Earth science and Biomedicine. In Earth system modeling, the model supports global weather forecasting, tropical cyclone (TC) forecast editing, and spatial downscaling driven by only language instructions. FuXi-Uni generates 10-day global forecasts at 0.25° resolution that outperform the SOTA physical forecasting system. It shows superior performance for both TC track and intensity prediction relative to the SOTA physical model, and generates high-resolution regional weather fields that surpass standard interpolation baselines. Regarding biomedicine, FuXi-Uni outperforms leading multimodal large language models on multiple biomedical visual question answering benchmarks. By unifying heterogeneous scientific modalities within a native shared latent space while maintaining strong domain-specific performance, FuXi-Uni provides a step forward more general-purpose, multimodal scientific models.

Related papers

• SciMIF: Understanding Multimodal Instruction Following in Scientific Domains [50.54744513908656]
  We introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions.<n>Based on an analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy.<n>We develop a high-fidelity instruction injection pipeline to systematically augment existing scientific datasets.
  arXiv  Detail & Related papers  (2026-08-26T16:30:20Z)
• Monkey King Bang: A Unified Scientific Multimodal Foundation Model [26.409792993729948]
  We introduce MKB, a unified scientific multimodal model for both understanding and generation.<n> MKB covers six scientific branches, including DNA, RNA, proteins, small molecules, earth science, and medical images.<n>It supports native outputs such as biological sequences, molecular strings, meteorological fields, and segmentation masks.
  arXiv  Detail & Related papers  (2026-07-17T13:56:50Z)
• S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation [31.901449470705618]
  We present S1- Omni, a unified multimodal reasoning model for scientific understanding, prediction, and generation.<n>The architecture of S1- Omni is built upon three core components: unified representation of scientific data, natural-world knowledge alignment, and decoding for domain-specific tasks.<n>S1- Omni is trained on S1- Omni-Corpus, which covers 200 scientific tasks and contains millions of reasoning samples, and is evaluated on over 60 scientific benchmarks.
  arXiv  Detail & Related papers  (2026-07-17T06:56:08Z)
• Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences [89.71422932447423]
  LOGOS is a scientific generative language model that unifies heterogeneous tasks across the natural sciences.<n>By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner.<n>Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines.
  arXiv  Detail & Related papers  (2026-06-15T16:14:53Z)
• DiffUNet^2: Bidirectional Prediction, Probabilistic Generation and Collaborative Visual Discovery for Scientific Data [25.698658042998073]
  temporal evolution is important to analyzing and reasoning about scientific phenomena.<n>Most machine learning methods provide deterministic forward predictions that overlook multiple plausible outcomes.<n>We present a framework that integrates diffusion-based generative modeling with interactive visual analytics for scientific exploration.
  arXiv  Detail & Related papers  (2026-06-02T17:15:01Z)
• WildSci: Advancing Scientific Reasoning from In-the-Wild Literature [50.16160754134139]
  We introduce WildSci, a new dataset of domain-specific science questions automatically synthesized from peer-reviewed literature.<n>By framing complex scientific reasoning tasks in a multiple-choice format, we enable scalable training with well-defined reward signals.<n>Experiments on a suite of scientific benchmarks demonstrate the effectiveness of our dataset and approach.
  arXiv  Detail & Related papers  (2026-01-09T06:35:23Z)
• HiSciBench: A Hierarchical Multi-disciplinary Benchmark for Scientific Intelligence from Reading to Discovery [50.8841471967624]
  HiSciBench is a hierarchical benchmark designed to evaluate foundation models across five levels that mirror the complete scientific workflow.<n>HiSciBench contains 8,735 carefully curated instances spanning six major scientific disciplines.
  arXiv  Detail & Related papers  (2025-12-28T12:08:05Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study [0.8633013637160062]
  We investigate the capability of several large language models to identify semantic relationships among research topics.<n>Our experiments demonstrate that fine-tuning Mess on PEM-Rel-8K yields excellent performance across all disciplines.
  arXiv  Detail & Related papers  (2025-08-28T11:53:45Z)
• SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications [63.92604046592333]
  Video foundation models (FMs) hold considerable promise as general-purpose domain-agnostic approaches.<n>We introduce SciVid, a benchmark comprising five tasks across medical computer vision, animal behavior, and weather forecasting.<n>We adapt six leading ViFMs to SciVid using simple trainable readout modules, establishing strong baselines and demonstrating potential for effective transfer learning.
  arXiv  Detail & Related papers  (2025-07-04T13:48:12Z)
• UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials [62.72989417755985]
  We present UniGenX, a unified generative model for function in natural systems.<n>UniGenX represents heterogeneous inputs as a mixed stream of symbolic and numeric tokens.<n>It achieves state-of-the-art or competitive performance for the function-aware generation across domains.
  arXiv  Detail & Related papers  (2025-03-09T16:43:07Z)
• Nature Language Model: Deciphering the Language of Nature for Scientific Discovery [105.55751854768297]
  Foundation models have revolutionized natural language processing and artificial intelligence.<n>We introduce Nature Language Model (NatureLM), a sequence-based science foundation model for scientific discovery.
  arXiv  Detail & Related papers  (2025-02-11T13:08:03Z)
• SciDFM: A Large Language Model with Mixture-of-Experts for Science [18.748699390397363]
  We introduce SciDFM, a mixture-of-experts LLM that is trained from scratch and is able to conduct college-level scientific reasoning. We collect a large-scale training corpus containing numerous scientific papers and books from different disciplines as well as data from domain-specific databases. We show that SciDFM achieves strong performance on general scientific benchmarks such as SciEval and SciQ, and it reaches a SOTA performance on domain-specific benchmarks among models of similar size.
  arXiv  Detail & Related papers  (2024-09-27T03:00:29Z)
• A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery [68.48094108571432]
  Large language models (LLMs) have revolutionized the way text and other modalities of data are handled. We aim to provide a more holistic view of the research landscape by unveiling cross-field and cross-modal connections between scientific LLMs.
  arXiv  Detail & Related papers  (2024-06-16T08:03:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.