Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data

• URL: http://arxiv.org/abs/2509.08679v1
• Date: Wed, 10 Sep 2025 15:19:04 GMT
• Title: Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data
• Authors: Jingya Cheng, Jiazi Tian, Federica Spoto, Alaleh Azhir, Daniel Mork, Hossein Estiri,
• Abstract summary: We develop a Signal Fidelity Index (SFI) diagnostic data quality at the patient level in dementia.<n>We test SFI-aware calibration for improving model performance across heterogeneous datasets without outcome labels.
• Score: 1.741250583668341
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: \textbf{Background:} Machine learning models trained on electronic health records (EHRs) often degrade across healthcare systems due to distributional shift. A fundamental but underexplored factor is diagnostic signal decay: variability in diagnostic quality and consistency across institutions, which affects the reliability of codes used for training and prediction. \textbf{Objective:} To develop a Signal Fidelity Index (SFI) quantifying diagnostic data quality at the patient level in dementia, and to test SFI-aware calibration for improving model performance across heterogeneous datasets without outcome labels. \textbf{Methods:} We built a simulation framework generating 2,500 synthetic datasets, each with 1,000 patients and realistic demographics, encounters, and coding patterns based on dementia risk factors. The SFI was derived from six interpretable components: diagnostic specificity, temporal consistency, entropy, contextual concordance, medication alignment, and trajectory stability. SFI-aware calibration applied a multiplicative adjustment, optimized across 50 simulation batches. \textbf{Results:} At the optimal parameter ($\alpha$ = 2.0), SFI-aware calibration significantly improved all metrics (p $<$ 0.001). Gains ranged from 10.3\% for Balanced Accuracy to 32.5\% for Recall, with notable increases in Precision (31.9\%) and F1-score (26.1\%). Performance approached reference standards, with F1-score and Recall within 1\% and Balanced Accuracy and Detection Rate improved by 52.3\% and 41.1\%, respectively. \textbf{Conclusions:} Diagnostic signal decay is a tractable barrier to model generalization. SFI-aware calibration provides a practical, label-free strategy to enhance prediction across healthcare contexts, particularly for large-scale administrative datasets lacking outcome labels.

Related papers

• Diagnostics for Individual-Level Prediction Instability in Machine Learning for Healthcare [0.0]
  We propose an evaluation framework that quantifies individual-level prediction instability by using two complementary diagnostics.<n>We apply these diagnostics to simulated data and GUSTO-I clinical dataset.
  arXiv  Detail & Related papers  (2026-02-27T03:42:28Z)
• Conformal Lesion Segmentation for 3D Medical Images [82.92159832699583]
  We propose a risk-constrained framework that calibrates data-driven thresholds via conformalization to ensure the test-time FNR remains below a target tolerance.<n>We validate the statistical soundness and predictive performance of CLS on six 3D-LS datasets across five backbone models, and conclude with actionable insights for deploying risk-aware segmentation in clinical practice.
  arXiv  Detail & Related papers  (2025-10-19T08:21:00Z)
• Hyperparameter Optimization and Reproducibility in Deep Learning Model Training [5.851295230237131]
  Reproducibility remains a critical challenge in foundation model training for histopathology.<n>We trained a CLIP model on the QUILT-1M dataset.<n>We identified clear trends: RandomResizedCrop values of 0.7-0.8 outperformed more aggressive (0.6) or conservative (0.9) settings.
  arXiv  Detail & Related papers  (2025-10-16T21:57:52Z)
• Advancing Tabular Stroke Modelling Through a Novel Hybrid Architecture and Feature-Selection Synergy [0.9999629695552196]
  The present work develops and validates a data-driven and interpretable machine-learning framework designed to predict strokes.<n>Ten routinely gathered demographic, lifestyle, and clinical variables were sourced from a public cohort of 4,981 records.<n>The proposed model achieved an accuracy rate of 97.2% and an F1-score of 97.15%, indicating a significant enhancement compared to the leading individual model.
  arXiv  Detail & Related papers  (2025-05-18T21:46:45Z)
• Graph-Based Fault Diagnosis for Rotating Machinery: Adaptive Segmentation and Structural Feature Integration [0.0]
  This paper proposes a graph-based framework for robust and interpretable multiclass fault diagnosis in rotating machinery.<n>It integrates entropy-optimized signal segmentation, time-frequency feature extraction, and graph-theoretic modeling to transform vibration signals into structured representations.<n>The proposed method achieves high diagnostic accuracy when evaluated on two benchmark datasets.
  arXiv  Detail & Related papers  (2025-04-29T13:34:52Z)
• Uncertainty-aware Long-tailed Weights Model the Utility of Pseudo-labels for Semi-supervised Learning [50.868594148443215]
  We propose an Uncertainty-aware Ensemble Structure (UES) to assess the utility of pseudo-labels for unlabeled samples.<n>UES is lightweight and architecture-agnostic, easily extending to various computer vision tasks, including classification and regression.
  arXiv  Detail & Related papers  (2025-03-13T02:21:04Z)
• EVolutionary Independent DEtermiNistiC Explanation [5.127310126394387]
  This paper introduces the Evolutionary Independent Deterministic Explanation (EVIDENCE) theory.<n>EVIDENCE offers a deterministic, model-independent method for extracting significant signals from black-box models.<n> Practical applications of EVIDENCE include improving diagnostic accuracy in healthcare and enhancing audio signal analysis.
  arXiv  Detail & Related papers  (2025-01-20T12:05:14Z)
• Machine Learning for ALSFRS-R Score Prediction: Making Sense of the Sensor Data [44.99833362998488]
  Amyotrophic Lateral Sclerosis (ALS) is a rapidly progressive neurodegenerative disease that presents individuals with limited treatment options. The present investigation, spearheaded by the iDPP@CLEF 2024 challenge, focuses on utilizing sensor-derived data obtained through an app.
  arXiv  Detail & Related papers  (2024-07-10T19:17:23Z)
• Towards Reliable Medical Image Segmentation by utilizing Evidential Calibrated Uncertainty [52.03490691733464]
  We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks. By leveraging subjective logic theory, we explicitly model probability and uncertainty for the problem of medical image segmentation. DeviS incorporates an uncertainty-aware filtering module, which utilizes the metric of uncertainty-calibrated error to filter reliable data.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• Ensemble model for pre-discharge icd10 coding prediction [45.82374977939355]
  We propose an ensemble model incorporating multiple clinical data sources for accurate code predictions. We obtain multi-label classification accuracies of 0.73 and 0.58 for average precision, 0.56 and 0.35 for F1-scores and 0.71 and 0.4 accuracy in predicting principal diagnosis for inpatient and outpatient datasets respectively.
  arXiv  Detail & Related papers  (2020-12-16T07:02:56Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.