Fact-Checking Generative AI: Ontology-Driven Biological Graphs for Disease-Gene Link Verification

• URL: http://arxiv.org/abs/2308.03929v4
• Date: Mon, 8 Apr 2024 14:47:37 GMT
• Title: Fact-Checking Generative AI: Ontology-Driven Biological Graphs for Disease-Gene Link Verification
• Authors: Ahmed Abdeen Hamed, Byung Suk Lee, Alessandro Crimi, Magdalena M. Misiak,
• Abstract summary: We aim to achieve fact-checking of the knowledge embedded in biological graphs that were contrived from ChatGPT contents. We adopted a biological networks approach that enables the systematic interrogation of ChatGPT's linked entities. This study demonstrated high accuracy of aggregate disease-gene links relationships found in ChatGPT-generated texts.
• Score: 45.65374554914359
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Since the launch of various generative AI tools, scientists have been striving to evaluate their capabilities and contents, in the hope of establishing trust in their generative abilities. Regulations and guidelines are emerging to verify generated contents and identify novel uses. we aspire to demonstrate how ChatGPT claims are checked computationally using the rigor of network models. We aim to achieve fact-checking of the knowledge embedded in biological graphs that were contrived from ChatGPT contents at the aggregate level. We adopted a biological networks approach that enables the systematic interrogation of ChatGPT's linked entities. We designed an ontology-driven fact-checking algorithm that compares biological graphs constructed from approximately 200,000 PubMed abstracts with counterparts constructed from a dataset generated using the ChatGPT-3.5 Turbo model. In 10-samples of 250 randomly selected records a ChatGPT dataset of 1000 "simulated" articles , the fact-checking link accuracy ranged from 70% to 86%. This study demonstrated high accuracy of aggregate disease-gene links relationships found in ChatGPT-generated texts.

Related papers

• DEMASQ: Unmasking the ChatGPT Wordsmith [63.8746084667206]
  We propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. Our method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machine-generated content and (ii) the alterations made by humans to evade previous detection methods.
  arXiv  Detail & Related papers  (2023-11-08T21:13:05Z)
• Assessing the Promise and Pitfalls of ChatGPT for Automated Code Generation [2.0400340435492272]
  This paper presents a comprehensive evaluation of the code generation capabilities of ChatGPT, a prominent large language model. A dataset of 131 code-generation prompts across 5 categories was curated to enable robust analysis. Code solutions were generated by both ChatGPT and humans for all prompts, resulting in 262 code samples.
  arXiv  Detail & Related papers  (2023-11-05T12:56:40Z)
• Applying BioBERT to Extract Germline Gene-Disease Associations for Building a Knowledge Graph from the Biomedical Literature [0.0]
  This paper presents SimpleGermKG, an automatic knowledge graph construction approach that connects germline genes and diseases. For the extraction of genes and diseases, we employ BioBERT, a pre-trained BERT model on biomedical corpora. For semantic relationships between articles, genes, and diseases, we implemented a part-whole relation approach. Our knowledge graph contains 297 genes, 130 diseases, and 46,747 triples.
  arXiv  Detail & Related papers  (2023-09-11T18:05:12Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Does Synthetic Data Generation of LLMs Help Clinical Text Mining? [51.205078179427645]
  We investigate the potential of OpenAI's ChatGPT to aid in clinical text mining. We propose a new training paradigm that involves generating a vast quantity of high-quality synthetic data. Our method has resulted in significant improvements in the performance of downstream tasks.
  arXiv  Detail & Related papers  (2023-03-08T03:56:31Z)
• The Digitalization of Bioassays in the Open Research Knowledge Graph [6.508148285794385]
  ORKG-assays is an AI micro-service written in Python designed to assist scientists obtain semantified bioassays as a set of triples. It uses an AI-based clustering algorithm which on gold-standard evaluations over 900 bioassays with 5,514 unique property-value pairs for 103 predicates shows competitive performance.
  arXiv  Detail & Related papers  (2022-03-28T08:35:01Z)
• A deep learning classifier for local ancestry inference [63.8376359764052]
  Local ancestry inference identifies the ancestry of each segment of an individual's genome. We develop a new LAI tool using a deep convolutional neural network with an encoder-decoder architecture. We show that our model is able to learn admixture as a zero-shot task, yielding ancestry assignments that are nearly as accurate as those from the existing gold standard tool, RFMix.
  arXiv  Detail & Related papers  (2020-11-04T00:42:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.