Why Do Probabilistic Clinical Models Fail To Transport Between Sites?

• URL: http://arxiv.org/abs/2311.04787v2
• Date: Thu, 28 Dec 2023 18:20:36 GMT
• Title: Why Do Probabilistic Clinical Models Fail To Transport Between Sites?
• Authors: Thomas A. Lasko, Eric V. Strobl, William W. Stead
• Abstract summary: Computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. We present common sources for this failure to transport, which we divide into sources under the control of the experimenter and sources inherent to the clinical data-generating process.
• Score: 6.660458629649825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rising popularity of artificial intelligence in healthcare is highlighting the problem that a computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. In this perspective, we present common sources for this failure to transport, which we divide into sources under the control of the experimenter and sources inherent to the clinical data-generating process. Of the inherent sources we look a little deeper into site-specific clinical practices that can affect the data distribution, and propose a potential solution intended to isolate the imprint of those practices on the data from the patterns of disease cause and effect that are the usual target of probabilistic clinical models.

Related papers

• A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR data [2.9033848132822726]
  We provide an accessible description of a generalizable causal machine learning pipeline to (i) discover latent causal sources of large-scale electronic health records observations, and (ii) quantify the source causal effects on clinical outcomes.<n>We illustrate how imperfect multimodal clinical data can be processed, decomposed into probabilistic independent latent sources, and used to train taskspecific causal models from which individual causal effects can be estimated.<n>We summarize the findings of the two real-world applications of the approach to date as a demonstration of its versatility and utility for medical discovery at scale.
  arXiv  Detail & Related papers  (2025-10-15T20:39:11Z)
• Causal Debiasing Medical Multimodal Representation Learning with Missing Modalities [6.02318066285653]
  Real-world medical datasets often suffer from missing modalities due to cost, protocol, or patient-specific constraints.<n>Our method consists of two key components: (1) a missingness deconfounding module that approximates causal intervention based on backdoor adjustment and (2) a dual-branch neural network that explicitly disentangles causal features from spurious correlations.
  arXiv  Detail & Related papers  (2025-09-06T06:27:10Z)
• Bridging the Generalisation Gap: Synthetic Data Generation for Multi-Site Clinical Model Validation [0.3362278589492841]
  Existing model evaluation approaches often rely on real-world datasets, which are limited in availability, embed confounding biases, and lack flexibility needed for systematic experimentation.<n>We propose a novel structured synthetic data framework designed for the controlled robustness of benchmarking model, fairness, and generalisability.
  arXiv  Detail & Related papers  (2025-04-29T11:04:28Z)
• Causal Lifting of Neural Representations: Zero-Shot Generalization for Causal Inferences [56.23412698865433]
  We focus on Prediction-Powered Causal Inferences (PPCI)<n> PPCI estimates the treatment effect in a target experiment with unlabeled factual outcomes, retrievable zero-shot from a pre-trained model.<n>We validate our method on synthetic and real-world scientific data, offering solutions to instances not solvable by vanilla Empirical Risk Minimization.
  arXiv  Detail & Related papers  (2025-02-10T10:52:17Z)
• Targeted Data Fusion for Causal Survival Analysis Under Distribution Shift [46.84912148188679]
  Causal inference has the potential to improve the generalizability, transportability, and replicability of scientific findings. Existing data fusion methods focus on binary or continuous outcomes. We propose two novel approaches for multi-source causal survival analysis.
  arXiv  Detail & Related papers  (2025-01-30T23:21:25Z)
• Empirical investigation of multi-source cross-validation in clinical ECG classification [0.9749684343406612]
  Cross-validation methods can be used to estimate the accuracy of machine learning-based clinical prediction models. We present a systematic empirical evaluation of standard K-fold cross-validation and leave-source-out cross-validation methods in a multi-source setting. Our results show that K-fold cross-validation, both on single-source and multi-source data, systemically overestimates prediction performance when the end goal is to generalize to new sources.
  arXiv  Detail & Related papers  (2024-03-22T07:56:31Z)
• Towards a Transportable Causal Network Model Based on Observational Healthcare Data [1.333879175460266]
  We propose a novel approach that combines selection diagrams, missingness graphs, causal discovery and prior knowledge into a single graphical model. We learn this model from data comprising two different cohorts of patients. The resulting causal network model is validated by expert clinicians in terms of risk assessment, accuracy and explainability.
  arXiv  Detail & Related papers  (2023-11-13T13:23:31Z)
• Domain-invariant Clinical Representation Learning by Bridging Data Distribution Shift across EMR Datasets [16.317118701435742]
  An effective prognostic model is expected to assist doctors in making right diagnosis and designing personalized treatment plan. In the early stage of a disease, limited data collection and clinical experiences, plus the concern out of privacy and ethics, may result in restricted data availability for reference. This article introduces a domain-invariant representation learning method to build a transition model from source dataset to target dataset.
  arXiv  Detail & Related papers  (2023-10-11T18:32:21Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• Federated learning for violence incident prediction in a simulated cross-institutional psychiatric setting [3.6939898792640213]
  Knowing who might become violent can influence staffing levels and severity. Machine learning models can assess each patient's likelihood of becoming violent based on clinical notes. Federated Learning can overcome the problem of data limitation by training models in a decentralised manner.
  arXiv  Detail & Related papers  (2022-05-17T07:37:12Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Trajectories, bifurcations and pseudotime in large clinical datasets: applications to myocardial infarction and diabetes data [94.37521840642141]
  We suggest a semi-supervised methodology for the analysis of large clinical datasets, characterized by mixed data types and missing values. The methodology is based on application of elastic principal graphs which can address simultaneously the tasks of dimensionality reduction, data visualization, clustering, feature selection and quantifying the geodesic distances (pseudotime) in partially ordered sequences of observations.
  arXiv  Detail & Related papers  (2020-07-07T21:04:55Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.