Learning Clinical Representations Under Systematic Distribution Shift

• URL: http://arxiv.org/abs/2603.07348v1
• Date: Sat, 07 Mar 2026 21:28:20 GMT
• Title: Learning Clinical Representations Under Systematic Distribution Shift
• Abstract summary: We propose a practice invariant representation learning framework for clinical prediction.<n>We combine supervised risk minimization with adversarial environment regularization and invariant risk penalties across hospitals.<n>Results demonstrate that explicitly accounting for systematic distribution shift during representation learning yields more robust transferable clinical models.
• Score: 1.7823291802496628
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical machine learning models are increasingly trained using large scale, multimodal foundation paradigms, yet deployment environments often differ systematically from the data generating settings used during training. Such shifts arise from heterogeneous measurement policies, documentation practices, and institutional workflows, leading to representation entanglement between physiologic signal and practice specific artifacts. In this work, we propose a practice invariant representation learning framework for multimodal clinical prediction. We model clinical observations as arising from latent physiologic factors and environment dependent processes, and introduce an objective that jointly optimizes predictive performance while suppressing environment predictive information in the learned embedding. Concretely, we combine supervised risk minimization with adversarial environment regularization and invariant risk penalties across hospitals. Across multiple longitudinal EHR prediction tasks and cross institution evaluations, our method improves out of distribution AUROC by up to 2 to 3 points relative to masked pretraining and standard supervised baselines, while maintaining in distribution performance and improving calibration. These results demonstrate that explicitly accounting for systematic distribution shift during representation learning yields more robust and transferable clinical models, highlighting the importance of structural invariance alongside architectural scale in healthcare AI.

Related papers

• From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction [61.12883122613684]
  This Review focuses on intervention-aware disease trajectory modeling in clinical AI--methods estimating patient-specific longitudinal disease evolution and assessing trajectory changes under alternative treatments.<n>We organize the field around six linked components: three decision tasks (factual forecasting, counterfactual estimation, policy evaluation) and three data-generating mechanisms (disease evolution, treatment assignment, observation process) that determine identifiability.<n>We present the first unified framework bridging forecasting, counterfactual trajectories, and policy evaluation across discrete/continuous time, explicitly addressing treatment assignment, time-varying confounding, and observation bias.
  arXiv  Detail & Related papers  (2026-05-16T10:45:26Z)
• Delving Aleatoric Uncertainty in Medical Image Segmentation via Vision Foundation Models [56.29123284262618]
  This work proposes leveraging the universal representation capabilities of visual foundation models to estimate inherent data uncertainty.<n>We analyze the feature diversity of the model's decoded representations and quantify their singular value energy to define the semantic perception scale for each class.<n>Based on this foundation, we design two uncertainty-driven application strategies: (1) the aleatoric uncertainty-aware data filtering mechanism to eliminate potentially noisy samples and enhance model learning quality; and (2) the dynamic uncertainty-aware optimization strategy that adaptively adjusts class-specific loss weights during training based on the semantic perception scale, combined with a label denoising mechanism to improve training stability
  arXiv  Detail & Related papers  (2026-04-13T03:59:54Z)
• Discriminative Representation Learning for Clinical Prediction [5.926309087813293]
  We argue that, when high quality supervision is available, direct outcome alignment may provide a stronger inductive bias than generative pretraining.<n>We propose a supervised deep learning framework that explicitly shapes representation geometry by maximizing inter class separation.<n>Our approach consistently outperforms masked, autoregressive, and contrastive pretraining baselines under matched model capacity.
  arXiv  Detail & Related papers  (2026-03-21T19:45:16Z)
• A Comedy of Estimators: On KL Regularization in RL Training of LLMs [81.7906270099878]
  reinforcement learning (RL) can substantially improve the reasoning performance of large language models (LLMs)<n>The RL objective for LLM training involves a regularization term, which is the reverse Kullback-Leibler (KL) divergence between the trained policy and the reference policy.<n>Recent works show that prevailing practices for incorporating KL regularization do not provide correct gradients for stated objectives, creating a discrepancy between the objective and its implementation.<n>We study the gradients of several estimators configurations, revealing how design choices shape gradient bias.
  arXiv  Detail & Related papers  (2025-12-26T04:20:58Z)
• Adaptive Test-Time Training for Predicting Need for Invasive Mechanical Ventilation in Multi-Center Cohorts [4.250825424649631]
  Test-Time Training (TTT) has emerged as a promising approach to mitigate such shifts.<n>We introduce Adaptive Test-Time Training (AdaTTT), an enhanced TTT framework tailored for EHR-based IMV prediction in ICU settings.
  arXiv  Detail & Related papers  (2025-12-07T04:27:40Z)
• Deep Learning Approach for Clinical Risk Identification Using Transformer Modeling of Heterogeneous EHR Data [0.0]
  This study proposes a Transformer-based longitudinal modeling method to address challenges in clinical risk classification with heterogeneous EHR data.<n>The method takes multi-source medical features as input and employs a feature embedding layer to achieve a unified representation of structured and unstructured data.<n> Experimental results show that the proposed model outperforms traditional machine learning and temporal deep learning models in accuracy, recall, precision, and F1-Score.
  arXiv  Detail & Related papers  (2025-11-06T08:02:21Z)
• Model Hemorrhage and the Robustness Limits of Large Language Models [119.46442117681147]
  Large language models (LLMs) demonstrate strong performance across natural language processing tasks, yet undergo significant performance degradation when modified for deployment.<n>We define this phenomenon as model hemorrhage - performance decline caused by parameter alterations and architectural changes.
  arXiv  Detail & Related papers  (2025-03-31T10:16:03Z)
• Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments [67.80453452949303]
  Estimating the conditional average treatment effect (CATE) from observational data is relevant for many applications such as personalized medicine. Here, we focus on the widespread setting where the observational data come from multiple environments. We propose different model-agnostic learners (so-called meta-learners) to estimate the bounds that can be used in combination with arbitrary machine learning models.
  arXiv  Detail & Related papers  (2024-06-04T16:31:43Z)
• Deployment of Image Analysis Algorithms under Prevalence Shifts [6.373765910269204]
  Domain gaps are among the most relevant roadblocks in the clinical translation of machine learning (ML)-based solutions for medical image analysis. We propose a workflow for prevalence-aware image classification that uses estimated deployment prevalences to adjust a trained classifier to a new environment.
  arXiv  Detail & Related papers  (2023-03-22T13:16:37Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)
• Variational Learning of Individual Survival Distributions [21.40142425105635]
  We introduce a variational time-to-event prediction model, named Variational Survival Inference (VSI), which builds upon recent advances in distribution learning techniques and deep neural networks. To validate effectiveness of our approach, an extensive set of experiments on both synthetic and real-world datasets is carried out, showing improved performance relative to competing solutions.
  arXiv  Detail & Related papers  (2020-03-09T22:09:51Z)
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks [103.14809802212535]
  We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions. Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions. To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
  arXiv  Detail & Related papers  (2020-02-27T18:46:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.