Clin-JEPA: A Multi-Phase Co-Training Framework for Joint-Embedding Predictive Pretraining on EHR Patient Trajectories

• URL: http://arxiv.org/abs/2605.10840v2
• Date: Tue, 12 May 2026 07:38:53 GMT
• Title: Clin-JEPA: A Multi-Phase Co-Training Framework for Joint-Embedding Predictive Pretraining on EHR Patient Trajectories
• Abstract summary: Clin-JEPA is a multi-phase co-training framework for predictive (JEPA) pretraining on EHR patient trajectories.<n>On MIMIC-IV ICU data, three independent evaluations support the framework.
• Score: 7.992399976910451
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Clin-JEPA, a multi-phase co-training framework for joint-embedding predictive (JEPA) pretraining on EHR patient trajectories. JEPA architectures have enabled latent-space planning in robotics and high-quality representation learning in vision, but extending the paradigm to EHR data -- to obtain a single backbone that simultaneously forecasts patient trajectories and serves diverse downstream risk-prediction tasks without per-task fine-tuning -- remains an open challenge. Existing JEPA frameworks either discard the predictor after pretraining (I-JEPA, V-JEPA) or train it on a frozen pretrained encoder (V-JEPA 2-AC), leaving the encoder unaware of the rollout signal that the retained predictor must use at inference; co-training the encoder and predictor under a shared JEPA prediction objective would supply this grounding, but naïve co-training is unstable, with representation collapse and online/target drift causing autoregressive rollout to diverge. Clin-JEPA's five-phase pretraining curriculum -- predictor warmup, joint refinement, EMA target alignment, hard sync, and predictor finalization -- addresses each failure mode by phase, stably co-training a Qwen3-8B-based encoder and a 92M-parameter latent trajectory predictor. On MIMIC-IV ICU data, three independent evaluations support the framework: (1) latent $\ell_1$ rollout drift uniquely converges ($-$15.7%) over 48-hour horizons while baselines and ablations diverge (+3% to +4951%); (2) the encoder learns a clinically discriminative latent geometry (deteriorating-patient cohorts displace 4.83$\times$ further than stable patients in latent space, vs $\leq$2.62$\times$ for baseline encoders); (3) a single backbone outperforms strong tabular and sequence baselines on multi-task downstream evaluation. Clin-JEPA achieves mean AUROC 0.851 on ICareFM EEP and 0.883 on 8 binary risk tasks (+0.038 and +0.041 vs baseline average).

Related papers

• On-Policy Self-Distillation in Diffusion Models [89.32656931795293]
  Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint rewards do not specify how an intermediate denoising prediction should change.<n>We introduce DiffusionOPSD as an on-policy self-distillation framework that converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries.<n>Across SD 3.5-M and the step-distilled Z-Image-Turbo, our approach achieves the best final held-out scores in 19 of 20 reward-matched settings across two backbones and ten evaluators.
  arXiv  Detail & Related papers  (2026-08-25T14:55:24Z)
• KMGen: A Skill-based Approach for Synthetic Individual Patient Data Generation [15.020554967749996]
  Individual patient data (IPD) from clinical trials is the substrate for survival modeling, meta-analysis, and safety research.<n>We introduce KMGen, the first end-to-end framework that automates KM curve extraction at accuracy competitive with human-guided tools.<n> KMGen generates synthetic per-patient AE trajectories from public trial registry records.
  arXiv  Detail & Related papers  (2026-08-23T21:43:01Z)
• An Embedded RISC-V Evaluation of Kolmogorov--Arnold Networks in Hard-Constrained Recurrent Physics-Informed Models [0.0]
  Kolmogorov--Arnold Networks (KANs) have been proposed as efficient replacements for multilayer perceptrons (MLPs)<n>This paper asks whether that parameter efficiency survives deployment.
  arXiv  Detail & Related papers  (2026-08-01T16:20:24Z)
• A Framework for Early Sepsis Prediction via Self-Supervised (JEPA) and Federated Representation Learning [0.0]
  Early sepsis prediction from electronic health records is challenged by irregular sampling, high missingness, and class imbalance.<n>We show that self-supervised pretraining with task-aware fine-tuning yields features that are both sharp near onset and robust across prediction horizons.
  arXiv  Detail & Related papers  (2026-07-18T07:27:04Z)
• LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning [59.34593956287438]
  We study how an SSL recipe behaves when its method-specific configuration is reused unchanged after the pretraining signal family changes.<n>We introduce Latent Euclidean Next-Embedding Prediction Architecture (LeNEPA)<n>LeNEPA replaces the stop-gradient/EMA stabilization used by vanilla NEPA with SIGReg-based isotropy regularization.
  arXiv  Detail & Related papers  (2026-07-01T13:56:21Z)
• A Cross-Layered Multi-Drone Coordination for Medical Supply Delivery during Disaster Response Management [0.45671221781968324]
  We present CEDA, a novel CTDE Deep Q-Network algorithm for cooperative multi-drone medical delivery.<n>CEDA jointly optimize triage-priority-aware routing, multi-agent coordination, and energy-efficient navigation under dynamic uncertainty.<n>CEDA achieves a delivery completion rate above 85%, reduces obstacle collisions by over 90% across training, and delivers an average of 6 patients per episode with a triage efficiency of 0.82.
  arXiv  Detail & Related papers  (2026-05-10T05:43:51Z)
• SPLICE: Latent Diffusion over JEPA Embeddings for Conformal Time-Series Inpainting [0.5076419064097734]
  Generative models for time-series imputation achieve strong reconstruction accuracy, yet provide no finite-sample reliability guarantees.<n>We introduce SPLICE, a modular framework coupling latent generative imputation with distribution-free, online-adaptive prediction intervals.<n>A pooled JEPA encoder trained on nine feeds transfers to four unseen domains, matching or exceeding per-dataset oracles with only a quick bridge fine-tuning.
  arXiv  Detail & Related papers  (2026-04-30T18:27:08Z)
• PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities [86.63247982275396]
  PRIME is a missing-aware multimodal self-supervised pretraining framework.<n>It learns robust and transferable representations from partially observed cohorts.<n>We evaluate PRIME on The Cancer Genome Atlas with label-free pretraining on 32 cancer types.
  arXiv  Detail & Related papers  (2026-04-05T21:14:27Z)
• Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing [79.88256756334327]
  Self-Distillation Policy Optimization (SDPO) addresses this by providing denser, more targeted logit-level supervision.<n>We propose Sample-Routed Policy Optimization (SRPO)<n>SRPO routes correct samples to GRPO's reward-aligned reinforcement and failed samples to SDPO's targeted logit-level correction.
  arXiv  Detail & Related papers  (2026-04-02T17:29:18Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics [53.247652209132376]
  Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D.<n>We present a comprehensive theory of JEPAs and instantiate it in bf LeJEPA, a lean, scalable, and theoretically grounded training objective.
  arXiv  Detail & Related papers  (2025-11-11T18:21:55Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• Label-Efficient Grasp Joint Prediction with Point-JEPA [0.0]
  3D self-supervised pretraining with Point--JEPA enables label-efficient grasp joint-angle prediction.<n>JEPA-style pretraining is a practical lever for data-efficient grasp learning.
  arXiv  Detail & Related papers  (2025-09-13T21:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.