Bias Evaluation and Mitigation in Retrieval-Augmented Medical Question-Answering Systems

• URL: http://arxiv.org/abs/2503.15454v3
• Date: Thu, 27 Mar 2025 01:51:23 GMT
• Title: Bias Evaluation and Mitigation in Retrieval-Augmented Medical Question-Answering Systems
• Authors: Yuelyu Ji, Hang Zhang, Yanshan Wang,
• Abstract summary: This study systematically evaluates demographic biases within medical RAG pipelines across multiple QA benchmarks.<n>We implement and compare several bias mitigation strategies to address identified biases, including Chain of Thought reasoning, Counterfactual filtering, Adversarial prompt refinement, and Majority Vote aggregation.
• Score: 4.031787614742573
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Medical Question Answering systems based on Retrieval Augmented Generation is promising for clinical decision support because they can integrate external knowledge, thus reducing inaccuracies inherent in standalone large language models (LLMs). However, these systems may unintentionally propagate or amplify biases associated with sensitive demographic attributes like race, gender, and socioeconomic factors. This study systematically evaluates demographic biases within medical RAG pipelines across multiple QA benchmarks, including MedQA, MedMCQA, MMLU, and EquityMedQA. We quantify disparities in retrieval consistency and answer correctness by generating and analyzing queries sensitive to demographic variations. We further implement and compare several bias mitigation strategies to address identified biases, including Chain of Thought reasoning, Counterfactual filtering, Adversarial prompt refinement, and Majority Vote aggregation. Experimental results reveal significant demographic disparities, highlighting that Majority Vote aggregation notably improves accuracy and fairness metrics. Our findings underscore the critical need for explicitly fairness-aware retrieval methods and prompt engineering strategies to develop truly equitable medical QA systems.

Related papers

• Uncertainty-aware abstention in medical diagnosis based on medical texts [87.88110503208016]
  This study addresses the critical issue of reliability for AI-assisted medical diagnosis.<n>We focus on the selection prediction approach that allows the diagnosis system to abstain from providing the decision if it is not confident in the diagnosis.<n>We introduce HUQ-2, a new state-of-the-art method for enhancing reliability in selective prediction tasks.
  arXiv  Detail & Related papers  (2025-02-25T10:15:21Z)
• MedCoT: Medical Chain of Thought via Hierarchical Expert [48.91966620985221]
  This paper presents MedCoT, a novel hierarchical expert verification reasoning chain method. It is designed to enhance interpretability and accuracy in biomedical imaging inquiries. Experimental evaluations on four standard Med-VQA datasets demonstrate that MedCoT surpasses existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-12-18T11:14:02Z)
• Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering [70.44269982045415]
  Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) We introduce Medical Retrieval-Augmented Generation Benchmark (MedRGB) that provides various supplementary elements to four medical QA datasets. Our experimental results reveals current models' limited ability to handle noise and misinformation in the retrieved documents.
  arXiv  Detail & Related papers  (2024-11-14T06:19:18Z)
• DiversityMedQA: Assessing Demographic Biases in Medical Diagnosis using Large Language Models [2.750784330885499]
  We introduce DiversityMedQA, a novel benchmark designed to assess large language models (LLMs) responses to medical queries across diverse patient demographics.<n>Our findings reveal notable discrepancies in model performance when tested against these demographic variations.
  arXiv  Detail & Related papers  (2024-09-02T23:37:20Z)
• A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models [20.11590976578911]
  Large language models (LLMs) hold promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evaluating equity-related model failures is a critical step toward developing systems that promote health equity. We present resources and methodologies for surfacing biases with potential to precipitate equity-related harms in long-form, LLM-generated answers to medical questions.
  arXiv  Detail & Related papers  (2024-03-18T17:56:37Z)
• Evaluating the Fairness of the MIMIC-IV Dataset and a Baseline Algorithm: Application to the ICU Length of Stay Prediction [65.268245109828]
  This paper uses the MIMIC-IV dataset to examine the fairness and bias in an XGBoost binary classification model predicting the ICU length of stay. The research reveals class imbalances in the dataset across demographic attributes and employs data preprocessing and feature extraction. The paper concludes with recommendations for fairness-aware machine learning techniques for mitigating biases and the need for collaborative efforts among healthcare professionals and data scientists.
  arXiv  Detail & Related papers  (2023-12-31T16:01:48Z)
• An AI-Guided Data Centric Strategy to Detect and Mitigate Biases in Healthcare Datasets [32.25265709333831]
  We generate a data-centric, model-agnostic, task-agnostic approach to evaluate dataset bias by investigating the relationship between how easily different groups are learned at small sample sizes (AEquity) We then apply a systematic analysis of AEq values across subpopulations to identify and manifestations of racial bias in two known cases in healthcare. AEq is a novel and broadly applicable metric that can be applied to advance equity by diagnosing and remediating bias in healthcare datasets.
  arXiv  Detail & Related papers  (2023-11-06T17:08:41Z)
• Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models [6.300835344100545]
  Leveraging artificial intelligence in conjunction with electronic health records holds transformative potential to improve healthcare. Yet, addressing bias in AI, which risks worsening healthcare disparities, cannot be overlooked. This study reviews methods to detect and mitigate diverse forms of bias in AI models developed using EHR data.
  arXiv  Detail & Related papers  (2023-10-30T18:29:15Z)
• Auditing ICU Readmission Rates in an Clinical Database: An Analysis of Risk Factors and Clinical Outcomes [0.0]
  This study presents a machine learning pipeline for clinical data classification in the context of a 30-day readmission problem. The fairness audit uncovers disparities in equal opportunity, predictive parity, false positive rate parity, and false negative rate parity criteria. The study suggests the need for collaborative efforts among researchers, policymakers, and practitioners to address bias and fairness in artificial intelligence (AI) systems.
  arXiv  Detail & Related papers  (2023-04-12T17:09:38Z)
• Fair Machine Learning in Healthcare: A Review [90.22219142430146]
  We analyze the intersection of fairness in machine learning and healthcare disparities. We provide a critical review of the associated fairness metrics from a machine learning standpoint. We propose several new research directions that hold promise for developing ethical and equitable ML applications in healthcare.
  arXiv  Detail & Related papers  (2022-06-29T04:32:10Z)
• Reinforcement Learning with Heterogeneous Data: Estimation and Inference [84.72174994749305]
  We introduce the K-Heterogeneous Markov Decision Process (K-Hetero MDP) to address sequential decision problems with population heterogeneity. We propose the Auto-Clustered Policy Evaluation (ACPE) for estimating the value of a given policy, and the Auto-Clustered Policy Iteration (ACPI) for estimating the optimal policy in a given policy class. We present simulations to support our theoretical findings, and we conduct an empirical study on the standard MIMIC-III dataset.
  arXiv  Detail & Related papers  (2022-01-31T20:58:47Z)
• Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management [5.044336341666555]
  We introduce Q-Pain, a dataset for assessing bias in medical QA in the context of pain management. We propose a new, rigorous framework, including a sample experimental design, to measure the potential biases present when making treatment decisions.
  arXiv  Detail & Related papers  (2021-08-03T21:55:28Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.