Leveraging Generative AI Through Prompt Engineering and Rigorous Validation to Create Comprehensive Synthetic Datasets for AI Training in Healthcare

• URL: http://arxiv.org/abs/2504.20921v1
• Date: Tue, 29 Apr 2025 16:37:34 GMT
• Title: Leveraging Generative AI Through Prompt Engineering and Rigorous Validation to Create Comprehensive Synthetic Datasets for AI Training in Healthcare
• Authors: Polycarp Nalela,
• Abstract summary: The GPT-4 API was employed to generate high-quality synthetic datasets aimed at overcoming this limitation.<n>The generated data encompassed a comprehensive array of patient admission information, including healthcare provider details, hospital departments, wards, bed assignments, patient demographics, emergency contacts, vital signs, immunizations, allergies, medical histories, appointments, hospital visits, laboratory tests, diagnoses, treatment plans, medications, clinical notes, visit logs, discharge summaries, and referrals.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Access to high-quality medical data is often restricted due to privacy concerns, posing significant challenges for training artificial intelligence (AI) algorithms within Electronic Health Record (EHR) applications. In this study, prompt engineering with the GPT-4 API was employed to generate high-quality synthetic datasets aimed at overcoming this limitation. The generated data encompassed a comprehensive array of patient admission information, including healthcare provider details, hospital departments, wards, bed assignments, patient demographics, emergency contacts, vital signs, immunizations, allergies, medical histories, appointments, hospital visits, laboratory tests, diagnoses, treatment plans, medications, clinical notes, visit logs, discharge summaries, and referrals. To ensure data quality and integrity, advanced validation techniques were implemented utilizing models such as BERT's Next Sentence Prediction for sentence coherence, GPT-2 for overall plausibility, RoBERTa for logical consistency, autoencoders for anomaly detection, and conducted diversity analysis. Synthetic data that met all validation criteria were integrated into a comprehensive PostgreSQL database, serving as the data management system for the EHR application. This approach demonstrates that leveraging generative AI models with rigorous validation can effectively produce high-quality synthetic medical data, facilitating the training of AI algorithms while addressing privacy concerns associated with real patient data.

Related papers

• Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunities [61.633126163190724]
  Mental illness is a widespread and debilitating condition with substantial societal and personal costs.<n>Recent advances in Artificial Intelligence (AI) hold great potential for recognizing and addressing conditions such as depression, anxiety disorder, bipolar disorder, schizophrenia, and post-traumatic stress disorder.<n>Privacy concerns, including the risk of sensitive data leakage from datasets and trained models, remain a critical barrier to deploying these AI systems in real-world clinical settings.
  arXiv  Detail & Related papers  (2025-02-01T15:10:02Z)
• Datasheets for Healthcare AI: A Framework for Transparency and Bias Mitigation [0.0]
  Bias, data incompleteness, and inaccuracies in training datasets can lead to unfair outcomes and amplify existing disparities.<n>We propose a dataset documentation framework that promotes transparency and ensures alignment with regulatory requirements.<n>The findings emphasise the importance of dataset documentation in fostering responsible AI development.
  arXiv  Detail & Related papers  (2025-01-09T23:36:34Z)
• A text-to-tabular approach to generate synthetic patient data using LLMs [0.3628457733531155]
  We propose an approach to generate synthetic patient data that does not require access to the original data.<n>We leverage prior medical knowledge and in-context learning capabilities of large language models to generate realistic patient data.
  arXiv  Detail & Related papers  (2024-12-06T16:10:40Z)
• Unlocking Historical Clinical Trial Data with ALIGN: A Compositional Large Language Model System for Medical Coding [44.01429184037945]
  We introduce ALIGN, a novel compositional LLM-based system for automated, zero-shot medical coding.<n>We evaluate ALIGN on harmonizing medication terms into Anatomical Therapeutic Chemical (ATC) and medical history terms into Medical Dictionary for Regulatory Activities (MedDRA) codes.
  arXiv  Detail & Related papers  (2024-11-20T09:59:12Z)
• TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasets [57.067409211231244]
  This paper presents meticulously curated AIready datasets covering multi-modal data (e.g., drug molecule, disease code, text, categorical/numerical features) and 8 crucial prediction challenges in clinical trial design. We provide basic validation methods for each task to ensure the datasets' usability and reliability. We anticipate that the availability of such open-access datasets will catalyze the development of advanced AI approaches for clinical trial design.
  arXiv  Detail & Related papers  (2024-06-30T09:13:10Z)
• Synthetic Data in Radiological Imaging: Current State and Future Outlook [3.047958668050099]
  Key challenge for the development and deployment of artificial intelligence (AI) solutions in radiology is solving the associated data limitations. In silico data offers a number of potential advantages to patient data, such as diminished patient harm, reduced cost, simplified data acquisition, scalability, improved quality assurance testing, and a mitigation approach to data imbalances.
  arXiv  Detail & Related papers  (2024-05-08T18:35:47Z)
• Generative AI-Driven Human Digital Twin in IoT-Healthcare: A Comprehensive Survey [53.691704671844406]
  The Internet of things (IoT) can significantly enhance the quality of human life, specifically in healthcare. The human digital twin (HDT) is proposed as an innovative paradigm that can comprehensively characterize the replication of the individual human body. HDT is envisioned to empower IoT-healthcare beyond the application of healthcare monitoring by acting as a versatile and vivid human digital testbed. Recently, generative artificial intelligence (GAI) may be a promising solution because it can leverage advanced AI algorithms to automatically create, manipulate, and modify valuable while diverse data.
  arXiv  Detail & Related papers  (2024-01-22T03:17:41Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Leveraging Generative AI Models for Synthetic Data Generation in Healthcare: Balancing Research and Privacy [0.0]
  generative AI models like GANs and VAEs offer a promising solution to balance valuable data access and patient privacy protection. In this paper, we examine generative AI models for creating realistic, anonymized patient data for research and training.
  arXiv  Detail & Related papers  (2023-05-09T08:12:44Z)
• Foresight -- Deep Generative Modelling of Patient Timelines using Electronic Health Records [46.024501445093755]
  Temporal modelling of medical history can be used to forecast and simulate future events, estimate risk, suggest alternative diagnoses or forecast complications. We present Foresight, a novel GPT3-based pipeline that uses NER+L tools (i.e. MedCAT) to convert document text into structured, coded concepts.
  arXiv  Detail & Related papers  (2022-12-13T19:06:00Z)
• Non-Imaging Medical Data Synthesis for Trustworthy AI: A Comprehensive Survey [6.277848092408045]
  Data quality is the key factor for the development of trustworthy AI in healthcare. Access to good quality datasets is limited by the technical difficulty of data acquisition. Large-scale sharing of healthcare data is hindered by strict ethical restrictions.
  arXiv  Detail & Related papers  (2022-09-17T13:34:17Z)
• The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation [81.72197368690031]
  We present a new benchmarking suite designed specifically for medical sequential decision making. The Medkit-Learn(ing) Environment is a publicly available Python package providing simple and easy access to high-fidelity synthetic medical data.
  arXiv  Detail & Related papers  (2021-06-08T10:38:09Z)
• DeepEnroll: Patient-Trial Matching with Deep Embedding and Entailment Prediction [67.91606509226132]
  Clinical trials are essential for drug development but often suffer from expensive, inaccurate and insufficient patient recruitment. DeepEnroll is a cross-modal inference learning model to jointly encode enrollment criteria (tabular data) into a shared latent space for matching inference.
  arXiv  Detail & Related papers  (2020-01-22T17:51:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.