Representation Before Training: A Fixed-Budget Benchmark for Generative Medical Event Models

• URL: http://arxiv.org/abs/2604.16775v1
• Date: Sat, 18 Apr 2026 01:38:47 GMT
• Title: Representation Before Training: A Fixed-Budget Benchmark for Generative Medical Event Models
• Abstract summary: We evaluate how representation decisions affect downstream prediction after a shared one-epoch pretraining budget.<n>We train 28 matched transformers on MIMIC-IV and evaluate them on 30 clinical outcomes.
• Score: 0.3250525349446657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Every prediction from a generative medical event model is bounded by how clinical events are tokenized, yet input representation is rarely isolated from other system and architectural choices. We evaluate how representation decisions affect downstream prediction after a shared one-epoch pretraining budget. We train 28 matched transformers on MIMIC-IV and evaluate them on 30 clinical outcomes in three experiments: (1) quantization granularity, reference-range anchoring, and code-value fusion; (2) value encoding (hard bins, soft discretization, code-normalized xVal) crossed with temporal encoding (event order, time tokens, admission-relative RoPE); and (3) native MIMIC laboratory/vital codes versus the Common Longitudinal ICU Format (CLIF)-remapped laboratory/vital codes with compression-preserving perturbation arms. In Experiment 1, fused code-value tokenization improves mortality AUROC from 0.891 to 0.915 (BH-adjusted p < 0.001), hospital length-of-stay AUROC from 0.763 to 0.788 (BH-adjusted p < 0.001), and, for the decile fused-vs-unfused comparison, mean regression Spearman rho across the 13 regression outcomes from 0.414 to 0.494. Across the three temporal encodings, event order only and admission-relative RoPE match or exceed inserting time tokens on average while shortening sequences by 11%. CLIF remapping preserves downstream performance in our single-site setting while yielding a smaller, clinically interpretable token set compatible with multi-site use. Finer-than-decile quantization, reference-range anchoring, and soft discretization help in selective outcomes, while code-normalized xVal remains well below the discrete and soft families, consistent with near-median suppression that persists after the affine variant.

Related papers

• INTERVenE: Temporal-Abstraction-Interval Based Transformers for Short-Horizon Medical Event Prediction [0.0]
  We present an interval-based, knowledge-based temporal abstraction (KBTA) that makes per-token attributions resolve to clinical concepts by construction.<n> evaluated on 57,078 MIMIC-IV admissions against GRU-D, STraTS, and KarmaLego.<n>An input-representation ablation confirms the lift transfers across structured discretizations, positioning KBTA-based intervals as the interpretable substrate.
  arXiv  Detail & Related papers  (2026-08-30T16:52:34Z)
• Subtraction-Based Tumor Segmentation and Lesion-Centered pCR Prediction for the MAMA-MIA Challenge [0.0]
  We describe the submission of team FME to the MAMA-MIA Challenge.<n>It evaluated primary tumor segmentation and prediction of pathological complete response.<n>FME ranked second in both tasks.
  arXiv  Detail & Related papers  (2026-08-29T09:23:45Z)
• A Multidimensional Data-Driven Hybrid Transformer Framework for Non-invasive Continuous Blood Pressure Prediction [28.58502100288247]
  We propose a hybrid Transformer framework to estimate diastolic and systolic BP from ECG/III feature sequences.<n>Rather than raw waveforms, the framework models 10-step sequences of physiological descriptors and two demographic cos.
  arXiv  Detail & Related papers  (2026-08-24T14:02:10Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• LLM-Extracted Covariates for Clinical Causal Inference: Rethinking Integration Strategies [7.5675661390024125]
  Causal inference from electronic health records is fundamentally limited by unmeasured confounding.<n>We show how to effectively integrate latent confounders into causal estimation pipelines.
  arXiv  Detail & Related papers  (2026-04-18T00:26:10Z)
• ConvFormer3D-TAP: Phase/Uncertainty-Aware Front-End Fusion for Cine CMR View Classification Pipelines [0.879504058268139]
  ConvFormer3D-TAP integrates 3D convolutional tokenization with multiscale self-attention to enhance robustness across cardiac phases and ambiguous temporal segments.<n>On a cohort of 150,974 clinically acquired cine sequences, ConvFormer3D- achieved 96% validation accuracy with per-class F1-scores.<n>These results support ConvFormer3DTAP as a scalable front-end for view routing, filtering and quality control in end-to-end cMRI.
  arXiv  Detail & Related papers  (2026-04-13T12:29:16Z)
• FlatASCEND: Autoregressive Clinical Sequence Generation with Continuous Time Prediction and Association-Based Pharmacological Testing [1.7865154997539017]
  We present FlatASCEND, a 14.5M- parameter autoregressive clinical sequence model using flat composite tokens and a zero-inflated log-normal time head.<n>A prompt-shuffle ablation shows patient-specific conditioning amplifies mechanistic pharmacological effects.<n>Generative evidence is strongest for short-horizon ICU data; outpatient temporal fidelity is weaker.
  arXiv  Detail & Related papers  (2026-04-13T11:43:51Z)
• Differential Attention-Augmented BiomedCLIP with Asymmetric Focal Optimization for Imbalanced Multi-Label Video Capsule Endoscopy Classification [1.2246649738388389]
  This work presents a multi-label classification framework for video capsule endoscopy (VCE)<n>It addresses the extreme class imbalance inherent in the Galar dataset through a combination of architectural and optimization-level strategies.
  arXiv  Detail & Related papers  (2026-03-18T16:04:50Z)
• PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation [51.96735866702332]
  PREBA is a retrieval-augmented framework that integrates PCA-weighted retrieval and Bayesian averaging aggregation.<n> PREBA significantly improves performance-for instance, reducing MAE by up to 40% and raising R2 from -0.13 to 0.62 over zero-shot inference.
  arXiv  Detail & Related papers  (2026-02-27T07:19:23Z)
• Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering [94.37535002230504]
  We develop a training-free, inference-time control framework termed Semantically Decoupled Latent Steering.<n>Our approach constructs a semantic-free intervention vector via large language model (LLM)-driven semantic decomposition.<n>We show that our approach significantly reduces the probability of historical hallucinations.
  arXiv  Detail & Related papers  (2026-02-27T04:49:01Z)
• Transparent Early ICU Mortality Prediction with Clinical Transformer and Per-Case Modality Attribution [42.85462513661566]
  We present a lightweight, transparent multimodal ensemble that fuses physiological time-series measurements with unstructured clinical notes from the first 48 hours of an ICU stay.<n>A logistic regression model combines predictions from two modality-specific models: a bidirectional LSTM for vitals and a finetuned ClinicalModernBERT transformer for notes.<n>On the MIMIC-III benchmark, our late-fusion ensemble improves discrimination over the best single model while maintaining well-calibrated predictions.
  arXiv  Detail & Related papers  (2025-11-19T20:11:49Z)
• Conformal Lesion Segmentation for 3D Medical Images [82.92159832699583]
  We propose a risk-constrained framework that calibrates data-driven thresholds via conformalization to ensure the test-time FNR remains below a target tolerance.<n>We validate the statistical soundness and predictive performance of CLS on six 3D-LS datasets across five backbone models, and conclude with actionable insights for deploying risk-aware segmentation in clinical practice.
  arXiv  Detail & Related papers  (2025-10-19T08:21:00Z)
• Clinical-Grade Blood Pressure Prediction in ICU Settings: An Ensemble Framework with Uncertainty Quantification and Cross-Institutional Validation [0.0]
  Blood pressure (BP) monitoring is critical in in tensive care units (ICUs) where hemodynamic instability can rapidly progress to cardiovascular collapse.<n>Current machine learning (ML) approaches suffer from three limitations: lack of external validation, absence of uncertainty quantification, and inadequate data leakage prevention.<n>This study presents the first comprehensive framework with novel algorithmic leakage prevention, uncertainty quantification, and cross-institutional validation.
  arXiv  Detail & Related papers  (2025-07-21T11:15:33Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• MSED: a multi-modal sleep event detection model for clinical sleep analysis [62.997667081978825]
  We designed a single deep neural network architecture to jointly detect sleep events in a polysomnogram. The performance of the model was quantified by F1, precision, and recall scores, and by correlating index values to clinical values.
  arXiv  Detail & Related papers  (2021-01-07T13:08:44Z)
• Ensemble model for pre-discharge icd10 coding prediction [45.82374977939355]
  We propose an ensemble model incorporating multiple clinical data sources for accurate code predictions. We obtain multi-label classification accuracies of 0.73 and 0.58 for average precision, 0.56 and 0.35 for F1-scores and 0.71 and 0.4 accuracy in predicting principal diagnosis for inpatient and outpatient datasets respectively.
  arXiv  Detail & Related papers  (2020-12-16T07:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.