Evaluating the Potential of Leading Large Language Models in Reasoning
Biology Questions
- URL: http://arxiv.org/abs/2311.07582v1
- Date: Sun, 5 Nov 2023 03:34:17 GMT
- Title: Evaluating the Potential of Leading Large Language Models in Reasoning
Biology Questions
- Authors: Xinyu Gong, Jason Holmes, Yiwei Li, Zhengliang Liu, Qi Gan, Zihao Wu,
Jianli Zhang, Yusong Zou, Yuxi Teng, Tian Jiang, Hongtu Zhu, Wei Liu,
Tianming Liu, Yajun Yan
- Abstract summary: This study evaluated the capabilities of leading Large Language Models (LLMs) in answering conceptual biology questions.
The models were tested on a 108-question multiple-choice exam covering biology topics in molecular biology, biological techniques, metabolic engineering, and synthetic biology.
The results indicated GPT-4's proficiency in logical reasoning and its potential to aid biology research through capabilities like data analysis, hypothesis generation, and knowledge integration.
- Score: 33.81650223615028
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advances in Large Language Models (LLMs) have presented new
opportunities for integrating Artificial General Intelligence (AGI) into
biological research and education. This study evaluated the capabilities of
leading LLMs, including GPT-4, GPT-3.5, PaLM2, Claude2, and SenseNova, in
answering conceptual biology questions. The models were tested on a
108-question multiple-choice exam covering biology topics in molecular biology,
biological techniques, metabolic engineering, and synthetic biology. Among the
models, GPT-4 achieved the highest average score of 90 and demonstrated the
greatest consistency across trials with different prompts. The results
indicated GPT-4's proficiency in logical reasoning and its potential to aid
biology research through capabilities like data analysis, hypothesis
generation, and knowledge integration. However, further development and
validation are still required before the promise of LLMs in accelerating
biological discovery can be realized.
Related papers
- Understanding Biology in the Age of Artificial Intelligence [4.299566787216408]
Modern life sciences research is increasingly relying on artificial intelligence approaches to model biological systems.
Although machine learning (ML) models are useful for identifying patterns in large, complex data sets, its widespread application in biological sciences represents a significant deviation from traditional methods of scientific inquiry.
Here, we identify general principles that can guide the design and application of ML systems to model biological phenomena and advance scientific knowledge.
arXiv Detail & Related papers (2024-03-06T23:20:34Z) - BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning [77.90250740041411]
This paper introduces BioT5+, an extension of the BioT5 framework, tailored to enhance biological research and drug discovery.
BioT5+ incorporates several novel features: integration of IUPAC names for molecular understanding, inclusion of extensive bio-text and molecule data from sources like bioRxiv and PubChem, the multi-task instruction tuning for generality across tasks, and a numerical tokenization technique for improved processing of numerical data.
arXiv Detail & Related papers (2024-02-27T12:43:09Z) - An Evaluation of Large Language Models in Bioinformatics Research [52.100233156012756]
We study the performance of large language models (LLMs) on a wide spectrum of crucial bioinformatics tasks.
These tasks include the identification of potential coding regions, extraction of named entities for genes and proteins, detection of antimicrobial and anti-cancer peptides, molecular optimization, and resolution of educational bioinformatics problems.
Our findings indicate that, given appropriate prompts, LLMs like GPT variants can successfully handle most of these tasks.
arXiv Detail & Related papers (2024-02-21T11:27:31Z) - Progress and Opportunities of Foundation Models in Bioinformatics [77.74411726471439]
Foundations models (FMs) have ushered in a new era in computational biology, especially in the realm of deep learning.
Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs.
Review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases.
arXiv Detail & Related papers (2024-02-06T02:29:17Z) - The Impact of Large Language Models on Scientific Discovery: a
Preliminary Study using GPT-4 [0.0]
This report focuses on GPT-4, the state-of-the-art language model.
We evaluate GPT-4's knowledge base, scientific understanding, scientific numerical calculation abilities, and various scientific prediction capabilities.
arXiv Detail & Related papers (2023-11-13T14:26:12Z) - ProBio: A Protocol-guided Multimodal Dataset for Molecular Biology Lab [67.24684071577211]
The challenge of replicating research results has posed a significant impediment to the field of molecular biology.
We first curate a comprehensive multimodal dataset, named ProBio, as an initial step towards this objective.
Next, we devise two challenging benchmarks, transparent solution tracking and multimodal action recognition, to emphasize the unique characteristics and difficulties associated with activity understanding in BioLab settings.
arXiv Detail & Related papers (2023-11-01T14:44:01Z) - Large Language Models, scientific knowledge and factuality: A systematic
analysis in antibiotic discovery [0.0]
This work examines the potential of Large Language Models for dialoguing with biomedical background knowledge.
Ten state-of-the-art models are tested in two prompting-based tasks: chemical compound definition generation and chemical compound-fungus relation determination.
Results show that while recent models have improved in fluency, factual accuracy is still low and models are biased towards over-represented entities.
arXiv Detail & Related papers (2023-05-28T22:46:21Z) - BioGPT: Generative Pre-trained Transformer for Biomedical Text
Generation and Mining [140.61707108174247]
We propose BioGPT, a domain-specific generative Transformer language model pre-trained on large scale biomedical literature.
We get 44.98%, 38.42% and 40.76% F1 score on BC5CDR, KD-DTI and DDI end-to-end relation extraction tasks respectively, and 78.2% accuracy on PubMedQA.
arXiv Detail & Related papers (2022-10-19T07:17:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.