Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation

• URL: http://arxiv.org/abs/2604.27014v1
• Date: Wed, 29 Apr 2026 11:32:59 GMT
• Title: Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation
• Abstract summary: The scarcity of high-quality annotated medical data poses a significant bottleneck for training robust machine learning models.<n>Privacy regulations restrict data sharing, making synthetic data generation a promising alternative.<n>The use of Large Language Models (LLMs) in a data augmentation pipeline could be leveraged as an alternative in this field.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The scarcity of high-quality annotated medical data, particularly in mental health, poses a significant bottleneck for training robust machine learning models. Privacy regulations restrict data sharing, making synthetic data generation a promising alternative. The use of Large Language Models (LLMs) in a data augmentation pipeline could be leveraged as an alternative in this field. In the proposed methodology, DeepSeek-R1, OpenBioLLM-Llama3 and Qwen 3.5 are used to generate synthetic mental health evaluation reports conditioned on specific International Classification of Diseases, Tenth Revision (ICD-10) codes. Because naive text generation can lead to mode collapse or privacy breaches (memorization), a comprehensive evaluation framework is introduced. The generated diagnostic texts are assessed across three dimensions: semantic fidelity, lexical diversity, and privacy/plagiarism. The results demonstrate that all models can generate clinically coherent, diverse, and privacy-safe synthetic reports, significantly expanding the available training data for clinical natural language processing tasks without compromising patient confidentiality.

Related papers

• PSyGenTAB: A Privacy-Preserving Framework for Synthetic Clinical Tabular Data Generation via Constrained Optimization [1.9205944025326402]
  We present PSyGenTAB, a framework for balancing privacy protection and clinical utility in synthetic healthcare data.<n>PSyGenTAB enforces minimum privacy thresholds while maximizing clinical data utility.<n>Downstream evaluation shows that models trained on synthetic data achieve performance comparable to those trained on real patient records.
  arXiv  Detail & Related papers  (2026-06-16T22:14:40Z)
• A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine [59.78991974851707]
  Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis.<n>Most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems.<n>We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications.
  arXiv  Detail & Related papers  (2026-01-29T18:48:21Z)
• Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning [49.559151128219725]
  Large language models (LLMs) have shown great promise in the medical domain, achieving strong performance on several benchmarks.<n>However, they continue to underperform in real-world medical scenarios, which often demand stronger context-awareness.<n>We propose Multifaceted Self-Refinement (MuSeR), a data-driven approach that enhances LLMs' context-awareness along three key facets.
  arXiv  Detail & Related papers  (2025-11-13T08:13:23Z)
• Revealing Interconnections between Diseases: from Statistical Methods to Large Language Models [0.15558822250482188]
  Identifying disease interconnections through manual analysis of large-scale clinical data is labor-intensive, subjective, and prone to expert disagreement.<n>We evaluate seven approaches for uncovering disease relationships based on two data sources: sequences of ICD-10 codes from MIMIC-IV EHRs and the full set of ICD-10 codes.<n>Our framework integrates the following: (i) a statistical co-occurrence analysis and a masked language modeling (MLM) approach using real clinical data; (ii) domain-specific BERT variants; and (iii) a general-purpose BERT and document retrieval.
  arXiv  Detail & Related papers  (2025-10-06T15:09:39Z)
• SynLLM: A Comparative Analysis of Large Language Models for Medical Tabular Synthetic Data Generation via Prompt Engineering [1.5020330976600738]
  We present SynLLM, a modular framework for generating high-quality synthetic medical data using open-source Large Language Models.<n>We evaluate SynLLM across three public medical datasets, including Diabetes, Cirrhosis, and Stroke.<n>Our results show that prompt engineering significantly impacts data quality and privacy risk, with rule-based prompts achieving the best privacy-quality balance.
  arXiv  Detail & Related papers  (2025-08-11T23:56:42Z)
• TheBlueScrubs-v1, a comprehensive curated medical dataset derived from the internet [1.4043931310479378]
  TheBlueScrubs-v1 is a curated dataset of over 25 billion medical tokens drawn from a broad-scale internet corpus.<n>Each text is assigned three LLM-based quality scores encompassing medical relevance, precision and factual detail, and safety and ethical standards.<n>This Data Descriptor details the dataset's creation and validation, underscoring its potential utility for medical AI research.
  arXiv  Detail & Related papers  (2025-04-01T22:25:19Z)
• Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling [6.193782515824411]
  We present a system for generating synthetic free-text medical records using Masked Language Modeling.<n>The system preserves critical medical information while introducing diversity in the generations and minimising re-identification risk.<n>The results demonstrate high-quality synthetic data with a HIPAA-compliant PHI recall rate of 96% and a re-identification risk of 3.5%.
  arXiv  Detail & Related papers  (2024-09-15T19:11:01Z)
• Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models [46.32860360019374]
  Large language models (LLMs) have shown promise in this domain, but their direct deployment can lead to privacy issues.<n>We propose an innovative, resource-efficient approach, ClinGen, which infuses knowledge into the process.<n>Our empirical study across 7 clinical NLP tasks and 16 datasets reveals that ClinGen consistently enhances performance across various tasks.
  arXiv  Detail & Related papers  (2023-11-01T04:37:28Z)
• Development and validation of a natural language processing algorithm to pseudonymize documents in the context of a clinical data warehouse [53.797797404164946]
  The study highlights the difficulties faced in sharing tools and resources in this domain. We annotated a corpus of clinical documents according to 12 types of identifying entities. We build a hybrid system, merging the results of a deep learning model as well as manual rules.
  arXiv  Detail & Related papers  (2023-03-23T17:17:46Z)
• Privacy-preserving medical image analysis [53.4844489668116]
  We present PriMIA, a software framework designed for privacy-preserving machine learning (PPML) in medical imaging. We show significantly better classification performance of a securely aggregated federated learning model compared to human experts on unseen datasets. We empirically evaluate the framework's security against a gradient-based model inversion attack.
  arXiv  Detail & Related papers  (2020-12-10T13:56:00Z)
• GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators [74.16405337436213]
  We propose Gradient-sanitized Wasserstein Generative Adrial Networks (GS-WGAN) GS-WGAN allows releasing a sanitized form of sensitive data with rigorous privacy guarantees. We find our approach consistently outperforms state-of-the-art approaches across multiple metrics.
  arXiv  Detail & Related papers  (2020-06-15T10:01:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.