Addressing Challenges in Data Quality and Model Generalization for Malaria Detection

• URL: http://arxiv.org/abs/2501.00464v1
• Date: Tue, 31 Dec 2024 14:25:55 GMT
• Title: Addressing Challenges in Data Quality and Model Generalization for Malaria Detection
• Authors: Kiswendsida Kisito Kabore, Desire Guel,
• Abstract summary: Malaria remains a significant global health burden, particularly in resource-limited regions where timely and accurate diagnosis is critical to effective treatment and control.<n>Deep Learning (DL) has emerged as a transformative tool for automating malaria detection and it offers high accuracy and scalability.<n>However, the effectiveness of these models is constrained by challenges in data quality and model generalization.<n>This article provides a comprehensive analysis of these challenges and their implications for malaria detection performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Malaria remains a significant global health burden, particularly in resource-limited regions where timely and accurate diagnosis is critical to effective treatment and control. Deep Learning (DL) has emerged as a transformative tool for automating malaria detection and it offers high accuracy and scalability. However, the effectiveness of these models is constrained by challenges in data quality and model generalization including imbalanced datasets, limited diversity and annotation variability. These issues reduce diagnostic reliability and hinder real-world applicability. This article provides a comprehensive analysis of these challenges and their implications for malaria detection performance. Key findings highlight the impact of data imbalances which can lead to a 20\% drop in F1-score and regional biases which significantly hinder model generalization. Proposed solutions, such as GAN-based augmentation, improved accuracy by 15-20\% by generating synthetic data to balance classes and enhance dataset diversity. Domain adaptation techniques, including transfer learning, further improved cross-domain robustness by up to 25\% in sensitivity. Additionally, the development of diverse global datasets and collaborative data-sharing frameworks is emphasized as a cornerstone for equitable and reliable malaria diagnostics. The role of explainable AI techniques in improving clinical adoption and trustworthiness is also underscored. By addressing these challenges, this work advances the field of AI-driven malaria detection and provides actionable insights for researchers and practitioners. The proposed solutions aim to support the development of accessible and accurate diagnostic tools, particularly for resource-constrained populations.

Related papers

• Attention-Based Synthetic Data Generation for Calibration-Enhanced Survival Analysis: A Case Study for Chronic Kidney Disease Using Electronic Health Records [1.7769033811751995]
  Masked Clinical Modelling (MCM) is an attention-based framework capable of generating high-fidelity synthetic datasets. MCM preserves critical clinical insights, such as hazard ratios, while enhancing survival model calibration.
  arXiv  Detail & Related papers  (2025-03-08T06:58:33Z)
• Fair Foundation Models for Medical Image Analysis: Challenges and Perspectives [2.5573554033525636]
  Foundation Models (FMs), trained on vast datasets through self-supervised learning, enable efficient adaptation across medical imaging tasks. These models demonstrate potential for enhancing fairness, though significant challenges remain in achieving consistent performance across demographic groups. This comprehensive framework advances current knowledge by demonstrating how systematic bias mitigation, combined with policy engagement, can effectively address both technical and institutional barriers to equitable AI in healthcare.
  arXiv  Detail & Related papers  (2025-02-24T04:54:49Z)
• Prediction and Detection of Terminal Diseases Using Internet of Medical Things: A Review [4.4389631374821255]
  AI-driven models have achieved over 98% accuracy in predicting heart disease, chronic kidney disease (CKD), Alzheimer's disease, and lung cancer. The incorporation of IoMT data, which is vast and heterogeneous, adds complexities in ensuring interoperability and security to protect patient privacy. Future research should focus on data standardization and advanced preprocessing techniques to improve data quality and interoperability.
  arXiv  Detail & Related papers  (2024-09-22T15:02:33Z)
• Crowdsourcing with Enhanced Data Quality Assurance: An Efficient Approach to Mitigate Resource Scarcity Challenges in Training Large Language Models for Healthcare [0.0]
  We propose a crowdsourcing framework enriched with quality control measures at the pre-, real-time, and post-data gathering stages. Our study evaluated the effectiveness of enhancing data quality through its impact on Large Language Models for predicting autism-related symptoms.
  arXiv  Detail & Related papers  (2024-05-16T08:29:00Z)
• Domain-invariant Clinical Representation Learning by Bridging Data Distribution Shift across EMR Datasets [16.317118701435742]
  An effective prognostic model is expected to assist doctors in making right diagnosis and designing personalized treatment plan. In the early stage of a disease, limited data collection and clinical experiences, plus the concern out of privacy and ethics, may result in restricted data availability for reference. This article introduces a domain-invariant representation learning method to build a transition model from source dataset to target dataset.
  arXiv  Detail & Related papers  (2023-10-11T18:32:21Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• Generative models improve fairness of medical classifiers under distribution shifts [49.10233060774818]
  We show that learning realistic augmentations automatically from data is possible in a label-efficient manner using generative models. We demonstrate that these learned augmentations can surpass ones by making models more robust and statistically fair in- and out-of-distribution.
  arXiv  Detail & Related papers  (2023-04-18T18:15:38Z)
• Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matching [49.78442796596806]
  We propose an innovative privacy-aware data augmentation approach for patient-trial matching (LLM-PTM) Our experiments demonstrate a 7.32% average improvement in performance using the proposed LLM-PTM method, and the generalizability to new data is improved by 12.12%.
  arXiv  Detail & Related papers  (2023-03-24T03:14:00Z)
• Data-Centric Epidemic Forecasting: A Survey [56.99209141838794]
  This survey delves into various data-driven methodological and practical advancements. We enumerate the large number of epidemiological datasets and novel data streams that are relevant to epidemic forecasting. We also discuss experiences and challenges that arise in real-world deployment of these forecasting systems.
  arXiv  Detail & Related papers  (2022-07-19T16:15:11Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.