Discriminative Representation Learning for Clinical Prediction

• URL: http://arxiv.org/abs/2603.20921v1
• Date: Sat, 21 Mar 2026 19:45:16 GMT
• Title: Discriminative Representation Learning for Clinical Prediction
• Abstract summary: We argue that, when high quality supervision is available, direct outcome alignment may provide a stronger inductive bias than generative pretraining.<n>We propose a supervised deep learning framework that explicitly shapes representation geometry by maximizing inter class separation.<n>Our approach consistently outperforms masked, autoregressive, and contrastive pretraining baselines under matched model capacity.
• Score: 5.926309087813293
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models in healthcare have largely adopted self supervised pretraining objectives inherited from natural language processing and computer vision, emphasizing reconstruction and large scale representation learning prior to downstream adaptation. We revisit this paradigm in outcome centric clinical prediction settings and argue that, when high quality supervision is available, direct outcome alignment may provide a stronger inductive bias than generative pretraining. We propose a supervised deep learning framework that explicitly shapes representation geometry by maximizing inter class separation relative to within class variance, thereby concentrating model capacity along clinically meaningful axes. Across multiple longitudinal electronic health record tasks, including mortality and readmission prediction, our approach consistently outperforms masked, autoregressive, and contrastive pretraining baselines under matched model capacity. The proposed method improves discrimination, calibration, and sample efficiency, while simplifying the training pipeline to a single stage optimization. These findings suggest that in low entropy, outcome driven healthcare domains, supervision can act as the statistically optimal driver of representation learning, challenging the assumption that large scale self supervised pretraining is a prerequisite for strong clinical performance.

Related papers

• Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models [38.37812950780674]
  We propose a reinforcement learning framework that treats EHR foundation models as generative policies over patient trajectories.<n>We formulate common clinical prediction problems as event-conditioned, time-windowed reasoning tasks.<n>We then design time-aware, rollout-sensitive rewards to account for finite rollout lengths and temporally inconclusive outcomes.
  arXiv  Detail & Related papers  (2026-09-10T23:10:38Z)
• On Predicting the Post-training Potential of Pre-trained LLMs [60.07459271263409]
  We introduce a new task of predicting post-training potential - forecasting a base model's performance before post-training.<n>We propose RuDE, a unified framework that bypasses the generation gap of base models by leveraging response discrimination.<n>Experiments demonstrate a correlation greater than 90% with post-training performance.
  arXiv  Detail & Related papers  (2026-05-12T11:33:49Z)
• Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models [53.15468578562038]
  We propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for Multimodal Large Language Models (MLLMs)<n>We first quantify the uncertainty from the model's failure to ground token predictions in the given image.<n>We then encourage more learning pressure on visually deficient tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples.
  arXiv  Detail & Related papers  (2026-05-06T13:08:12Z)
• Learning Clinical Representations Under Systematic Distribution Shift [1.7823291802496628]
  We propose a practice invariant representation learning framework for clinical prediction.<n>We combine supervised risk minimization with adversarial environment regularization and invariant risk penalties across hospitals.<n>Results demonstrate that explicitly accounting for systematic distribution shift during representation learning yields more robust transferable clinical models.
  arXiv  Detail & Related papers  (2026-03-07T21:28:20Z)
• Dense Feature Learning via Linear Structure Preservation in Medical Data [30.77691570199694]
  Deep learning models for medical data are typically trained using task specific objectives that encourage representations to collapse onto a small number of discriminative directions.<n>We propose dense feature learning, a representation centric framework that explicitly shapes the linear structure of medical embeddings.
  arXiv  Detail & Related papers  (2026-02-07T21:23:35Z)
• Adaptive Test-Time Training for Predicting Need for Invasive Mechanical Ventilation in Multi-Center Cohorts [4.250825424649631]
  Test-Time Training (TTT) has emerged as a promising approach to mitigate such shifts.<n>We introduce Adaptive Test-Time Training (AdaTTT), an enhanced TTT framework tailored for EHR-based IMV prediction in ICU settings.
  arXiv  Detail & Related papers  (2025-12-07T04:27:40Z)
• Will Large Language Models Transform Clinical Prediction? [6.239284099493876]
  Large language models (LLMs) are attracting increasing interest in healthcare.<n>This commentary evaluates the potential of LLMs to improve clinical prediction models (CPMs) for diagnostic and prognostic tasks.
  arXiv  Detail & Related papers  (2025-05-23T17:02:04Z)
• In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention [52.159541540613915]
  We study how multi-head softmax attention models are trained to perform in-context learning on linear data.<n>Our results reveal that in-context learning ability emerges from the trained transformer as an aggregated effect of its architecture and the underlying data distribution.
  arXiv  Detail & Related papers  (2025-03-17T02:00:49Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images [71.26717896083433]
  Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging.
  arXiv  Detail & Related papers  (2023-09-05T19:45:09Z)
• Self-Distillation for Further Pre-training of Transformers [83.84227016847096]
  We propose self-distillation as a regularization for a further pre-training stage. We empirically validate the efficacy of self-distillation on a variety of benchmark datasets for image and text classification tasks.
  arXiv  Detail & Related papers  (2022-09-30T02:25:12Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.