BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoning

• URL: http://arxiv.org/abs/2502.16660v2
• Date: Thu, 27 Feb 2025 17:17:08 GMT
• Title: BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoning
• Authors: Haiteng Zhao, Chang Ma, Fangzhi Xu, Lingpeng Kong, Zhi-Hong Deng,
• Abstract summary: We introduce BioMaze, a dataset with 5.1K complex pathway problems from real research.<n>Our evaluation of methods such as CoT and graph-augmented reasoning, shows that LLMs struggle with pathway reasoning.<n>To address this, we propose PathSeeker, an LLM agent that enhances reasoning through interactive subgraph-based navigation.
• Score: 49.487327661584686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The applications of large language models (LLMs) in various biological domains have been explored recently, but their reasoning ability in complex biological systems, such as pathways, remains underexplored, which is crucial for predicting biological phenomena, formulating hypotheses, and designing experiments. This work explores the potential of LLMs in pathway reasoning. We introduce BioMaze, a dataset with 5.1K complex pathway problems derived from real research, covering various biological contexts including natural dynamic changes, disturbances, additional intervention conditions, and multi-scale research targets. Our evaluation of methods such as CoT and graph-augmented reasoning, shows that LLMs struggle with pathway reasoning, especially in perturbed systems. To address this, we propose PathSeeker, an LLM agent that enhances reasoning through interactive subgraph-based navigation, enabling a more effective approach to handling the complexities of biological systems in a scientifically aligned manner. The dataset and code are available at https://github.com/zhao-ht/BioMaze.

Related papers

• Large Language Models for Zero-shot Inference of Causal Structures in Biology [4.650342334505084]
  We present a framework to evaluate large language models (LLMs) for zero-shot inference of causal relationships in biology. We systematically evaluate causal claims obtained from an LLM using real-world interventional data. Our results show that even relatively small LLMs can capture meaningful aspects of causal structure in biological systems.
  arXiv  Detail & Related papers  (2025-03-06T11:43:30Z)
• Biological Sequence with Language Model Prompting: A Survey [14.270959261105968]
  Large Language models (LLMs) have emerged as powerful tools for addressing challenges across diverse domains. This paper systematically investigates the application of prompt-based methods with LLMs to biological sequences.
  arXiv  Detail & Related papers  (2025-03-06T06:28:36Z)
• Contextualizing biological perturbation experiments through language [3.704686482174365]
  PerturbQA is a benchmark for structured reasoning over perturbation experiments. We evaluate state-of-the-art machine learning and statistical approaches for modeling perturbations. As a proof of feasibility, we introduce Summer (SUMMarize, retrievE, and answeR), a simple, domain-informed LLM framework.
  arXiv  Detail & Related papers  (2025-02-28T18:15:31Z)
• ExPath: Towards Explaining Targeted Pathways for Biological Knowledge Bases [36.89299758497499]
  We propose a novel pathway inference framework, ExPath, to classify various graphs (bio-networks) in biological databases. ExPath comprises three components: (1) a large protein language model (pLM) that encodes and embeds AA-seqs into graph, overcoming traditional obstacles in processing AA-seq data; (2) PathMamba, a hybrid architecture combining graph neural networks (GNNs) with state-space sequence modeling (Mamba) to capture both local interactions and global pathway-level dependencies; and (3) PathExplainer, a subgraph learning module that identifies functionally critical nodes and edges through train
  arXiv  Detail & Related papers  (2025-02-25T09:33:15Z)
• Biology Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models [51.316001071698224]
  We introduce Biology-Instructions, the first large-scale multi-omics biological sequences-related instruction-tuning dataset.<n>This dataset can bridge the gap between large language models (LLMs) and complex biological sequences-related tasks.<n>We also develop a strong baseline called ChatMultiOmics with a novel three-stage training pipeline.
  arXiv  Detail & Related papers  (2024-12-26T12:12:23Z)
• Causal Representation Learning from Multimodal Biological Observations [57.00712157758845]
  We aim to develop flexible identification conditions for multimodal data. We establish identifiability guarantees for each latent component, extending the subspace identification results from prior work. Our key theoretical ingredient is the structural sparsity of the causal connections among distinct modalities.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• A Review of BioTree Construction in the Context of Information Fusion: Priors, Methods, Applications and Trends [41.740569399988644]
  Biological tree (BioTree) analysis is a foundational tool in biology, enabling the exploration of evolutionary and differentiation.<n>Traditional tree construction methods face challenges in handling the growing complexity and scale of modern biological data.<n>Advances in deep learning (DL) offer transformative opportunities by enabling the fusion of biological prior knowledge with data-driven models.
  arXiv  Detail & Related papers  (2024-10-07T08:00:41Z)
• An Evaluation of Large Language Models in Bioinformatics Research [52.100233156012756]
  We study the performance of large language models (LLMs) on a wide spectrum of crucial bioinformatics tasks. These tasks include the identification of potential coding regions, extraction of named entities for genes and proteins, detection of antimicrobial and anti-cancer peptides, molecular optimization, and resolution of educational bioinformatics problems. Our findings indicate that, given appropriate prompts, LLMs like GPT variants can successfully handle most of these tasks.
  arXiv  Detail & Related papers  (2024-02-21T11:27:31Z)
• Progress and Opportunities of Foundation Models in Bioinformatics [77.74411726471439]
  Foundations models (FMs) have ushered in a new era in computational biology, especially in the realm of deep learning. Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs. Review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases.
  arXiv  Detail & Related papers  (2024-02-06T02:29:17Z)
• ProBio: A Protocol-guided Multimodal Dataset for Molecular Biology Lab [67.24684071577211]
  The challenge of replicating research results has posed a significant impediment to the field of molecular biology. We first curate a comprehensive multimodal dataset, named ProBio, as an initial step towards this objective. Next, we devise two challenging benchmarks, transparent solution tracking and multimodal action recognition, to emphasize the unique characteristics and difficulties associated with activity understanding in BioLab settings.
  arXiv  Detail & Related papers  (2023-11-01T14:44:01Z)
• Causal machine learning for single-cell genomics [94.28105176231739]
  We discuss the application of machine learning techniques to single-cell genomics and their challenges. We first present the model that underlies most of current causal approaches to single-cell biology. We then identify open problems in the application of causal approaches to single-cell data.
  arXiv  Detail & Related papers  (2023-10-23T13:35:24Z)
• Active Inference Tree Search in Large POMDPs [0.0]
  We introduce a novel method to plan in POMDPs--Active Inference Tree Search (AcT)<n>AcT combines the normative character and biological realism of a leading planning theory in neuroscience (Active Inference) and the scalability of tree search methods in AI.<n>Our simulations show that AcT successfully navigates binary trees that are challenging for sampling-based methods, problems that require adaptive exploration, and the large POMDP problem 'RockSample'--in which AcT reproduces state-of-the-art POMDP solutions.
  arXiv  Detail & Related papers  (2021-03-25T14:17:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.