Evaluation of large language models for discovery of gene set function

• URL: http://arxiv.org/abs/2309.04019v2
• Date: Mon, 1 Apr 2024 05:02:23 GMT
• Title: Evaluation of large language models for discovery of gene set function
• Authors: Mengzhou Hu, Sahar Alkhairy, Ingoo Lee, Rudolf T. Pillich, Dylan Fong, Kevin Smith, Robin Bachelder, Trey Ideker, Dexter Pratt,
• Abstract summary: We evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set. Benchmarking against canonical gene sets from the Gene Ontology, GPT-4 confidently recovered the curated name or a more general concept. In gene sets derived from 'omics data, GPT-4 identified novel functions not reported by classical functional enrichment.
• Score: 0.8864741602534821
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Gene set analysis is a mainstay of functional genomics, but it relies on curated databases of gene functions that are incomplete. Here we evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set, substantiated by supporting rationale, citations and a confidence assessment. Benchmarking against canonical gene sets from the Gene Ontology, GPT-4 confidently recovered the curated name or a more general concept (73% of cases), while benchmarking against random gene sets correctly yielded zero confidence. Gemini-Pro and Mixtral-Instruct showed ability in naming but were falsely confident for random sets, whereas Llama2-70b had poor performance overall. In gene sets derived from 'omics data, GPT-4 identified novel functions not reported by classical functional enrichment (32% of cases), which independent review indicated were largely verifiable and not hallucinations. The ability to rapidly synthesize common gene functions positions LLMs as valuable 'omics assistants.

Related papers

• Predicting Genetic Mutation from Whole Slide Images via Biomedical-Linguistic Knowledge Enhanced Multi-label Classification [119.13058298388101]
  We develop a Biological-knowledge enhanced PathGenomic multi-label Transformer to improve genetic mutation prediction performances. BPGT first establishes a novel gene encoder that constructs gene priors by two carefully designed modules. BPGT then designs a label decoder that finally performs genetic mutation prediction by two tailored modules.
  arXiv  Detail & Related papers  (2024-06-05T06:42:27Z)
• BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments [112.25067497985447]
  We introduce BioDiscoveryAgent, an agent that designs new experiments, reasons about their outcomes, and efficiently navigates the hypothesis space to reach desired solutions. BioDiscoveryAgent can uniquely design new experiments without the need to train a machine learning model. It achieves an average of 21% improvement in predicting relevant genetic perturbations across six datasets.
  arXiv  Detail & Related papers  (2024-05-27T19:57:17Z)
• GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases [5.831842925038342]
  We present GeneAgent, a first-of-its-kind language agent featuring self-verification capability. It autonomously interacts with various biological databases to improve accuracy and reduce hallucination occurrences. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms standard GPT-4 by a significant margin.
  arXiv  Detail & Related papers  (2024-05-25T12:35:15Z)
• VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling [60.91599380893732]
  VQDNA is a general-purpose framework that renovates genome tokenization from the perspective of genome vocabulary learning. By leveraging vector-quantized codebooks as learnable vocabulary, VQDNA can adaptively tokenize genomes into pattern-aware embeddings.
  arXiv  Detail & Related papers  (2024-05-13T20:15:03Z)
• Efficient and Scalable Fine-Tune of Language Models for Genome Understanding [49.606093223945734]
  We present textscLingo: textscLanguage prefix ftextscIne-tuning for textscGentextscOmes. Unlike DNA foundation models, textscLingo strategically leverages natural language foundation models' contextual cues. textscLingo further accommodates numerous downstream fine-tune tasks by an adaptive rank sampling method.
  arXiv  Detail & Related papers  (2024-02-12T21:40:45Z)
• ProtiGeno: a prokaryotic short gene finder using protein language models [1.2354076490479513]
  Current gene finders are highly sensitive in finding long genes, but their sensitivity decreases noticeably in finding shorter genes. We develop a deep learning-based method called ProtiGeno, specifically targeting short prokaryotic genes. In systematic large-scale experiments on 4,288 prokaryotic genomes, we demonstrate that ProtiGeno predicts short coding and noncoding genes with higher accuracy and recall than the current state-of-the-art gene finders.
  arXiv  Detail & Related papers  (2023-07-19T16:46:42Z)
• Gene Set Summarization using Large Language Models [1.312659265502151]
  We develop a method that uses GPT models to perform gene set function summarization. We demonstrate that these methods are able to generate plausible and biologically valid summary GO term lists for gene sets. However, GPT-based approaches are unable to deliver reliable scores or p-values and often return terms that are not statistically significant.
  arXiv  Detail & Related papers  (2023-05-21T02:06:33Z)
• Machine Learning Methods for Cancer Classification Using Gene Expression Data: A Review [77.34726150561087]
  Cancer is the second major cause of death after cardiovascular diseases. Gene expression can play a fundamental role in the early detection of cancer. This study reviews recent progress in gene expression analysis for cancer classification using machine learning methods.
  arXiv  Detail & Related papers  (2023-01-28T15:03:03Z)
• Feature extraction using Spectral Clustering for Gene Function Prediction [0.4492444446637856]
  This paper presents a novel in silico approach for to the annotation problem that combines cluster analysis and hierarchical multi-label classification. The proposed approach is applied to a case study on Zea mays, one of the most dominant and productive crops in the world.
  arXiv  Detail & Related papers  (2022-03-25T10:17:36Z)
• Cancer Gene Profiling through Unsupervised Discovery [49.28556294619424]
  We introduce a novel, automatic and unsupervised framework to discover low-dimensional gene biomarkers. Our method is based on the LP-Stability algorithm, a high dimensional center-based unsupervised clustering algorithm. Our signature reports promising results on distinguishing immune inflammatory and immune desert tumors.
  arXiv  Detail & Related papers  (2021-02-11T09:04:45Z)
• Mining Functionally Related Genes with Semi-Supervised Learning [0.0]
  We introduce a rich set of features and use them in conjunction with semisupervised learning approaches. The framework of learning with positive and unlabeled examples (LPU) is shown to be especially appropriate for mining functionally related genes.
  arXiv  Detail & Related papers  (2020-11-05T20:34:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.