Can ChatGPT be Your Personal Medical Assistant?
- URL: http://arxiv.org/abs/2312.12006v1
- Date: Tue, 19 Dec 2023 09:54:27 GMT
- Title: Can ChatGPT be Your Personal Medical Assistant?
- Authors: Md. Rafiul Biswas, Ashhadul Islam, Zubair Shah, Wajdi Zaghouani, Samir
Brahim Belhaouari
- Abstract summary: This study uses publicly available online questions and answering datasets in Arabic language.
There are almost 430K questions and answers for 20 disease-specific categories.
The performance of this fine-tuned model was evaluated through automated and human evaluation.
- Score: 0.09264362806173355
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The advanced large language model (LLM) ChatGPT has shown its potential in
different domains and remains unbeaten due to its characteristics compared to
other LLMs. This study aims to evaluate the potential of using a fine-tuned
ChatGPT model as a personal medical assistant in the Arabic language. To do so,
this study uses publicly available online questions and answering datasets in
Arabic language. There are almost 430K questions and answers for 20
disease-specific categories. GPT-3.5-turbo model was fine-tuned with a portion
of this dataset. The performance of this fine-tuned model was evaluated through
automated and human evaluation. The automated evaluations include perplexity,
coherence, similarity, and token count. Native Arabic speakers with medical
knowledge evaluated the generated text by calculating relevance, accuracy,
precision, logic, and originality. The overall result shows that ChatGPT has a
bright future in medical assistance.
Related papers
- Enhancing Medical Support in the Arabic Language Through Personalized ChatGPT Assistance [1.174020933567308]
ChatGPT provides real-time, personalized medical diagnosis at no cost.
The study involved compiling a dataset of disease information and generating multiple messages for each disease.
ChatGPT's performance was assessed by measuring the similarity between its responses and the actual diseases.
arXiv Detail & Related papers (2024-03-21T21:28:07Z) - Performance of ChatGPT on USMLE: Unlocking the Potential of Large
Language Models for AI-Assisted Medical Education [0.0]
This study determined how reliable ChatGPT can be for answering complex medical and clinical questions.
The paper evaluated the obtained results using a 2-way ANOVA and posthoc analysis.
ChatGPT-generated answers were found to be more context-oriented than regular Google search results.
arXiv Detail & Related papers (2023-06-30T19:53:23Z) - HuatuoGPT, towards Taming Language Model to Be a Doctor [67.96794664218318]
HuatuoGPT is a large language model (LLM) for medical consultation.
We leverage both textitdistilled data from ChatGPT and textitreal-world data from doctors in the supervised fine-tuned stage.
arXiv Detail & Related papers (2023-05-24T11:56:01Z) - ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large
Language Models in Multilingual Learning [70.57126720079971]
Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP)
This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources.
Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
arXiv Detail & Related papers (2023-04-12T05:08:52Z) - ChatGPT-Crawler: Find out if ChatGPT really knows what it's talking
about [15.19126287569545]
This research examines the responses generated by ChatGPT from different Conversational QA corpora.
The study employed BERT similarity scores to compare these responses with correct answers and obtain Natural Language Inference(NLI) labels.
The study identified instances where ChatGPT provided incorrect answers to questions, providing insights into areas where the model may be prone to error.
arXiv Detail & Related papers (2023-04-06T18:42:47Z) - To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection.
We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains.
Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
arXiv Detail & Related papers (2023-04-04T03:04:28Z) - A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on
Reasoning, Hallucination, and Interactivity [79.12003701981092]
We carry out an extensive technical evaluation of ChatGPT using 23 data sets covering 8 different common NLP application tasks.
We evaluate the multitask, multilingual and multi-modal aspects of ChatGPT based on these data sets and a newly designed multimodal dataset.
ChatGPT is 63.41% accurate on average in 10 different reasoning categories under logical reasoning, non-textual reasoning, and commonsense reasoning.
arXiv Detail & Related papers (2023-02-08T12:35:34Z) - Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot.
Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community.
It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
arXiv Detail & Related papers (2023-02-08T09:44:51Z) - A Categorical Archive of ChatGPT Failures [47.64219291655723]
ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation.
It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries.
However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.
arXiv Detail & Related papers (2023-02-06T04:21:59Z) - How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation,
and Detection [8.107721810172112]
ChatGPT is able to respond effectively to a wide range of human questions.
People are starting to worry about the potential negative impacts that large language models (LLMs) like ChatGPT could have on society.
In this work, we collected tens of thousands of comparison responses from both human experts and ChatGPT.
arXiv Detail & Related papers (2023-01-18T15:23:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.