Rethinking Tokenization for Clinical Time Series: When Less is More

• URL: http://arxiv.org/abs/2512.05217v1
• Date: Thu, 04 Dec 2025 19:39:10 GMT
• Title: Rethinking Tokenization for Clinical Time Series: When Less is More
• Abstract summary: We present a systematic evaluation of tokenization approaches for clinical time series modeling.<n>We show that explicit time encodings provide no consistent statistically significant benefit for the evaluated downstream tasks.<n>Value features show task-dependent importance, affecting mortality prediction but not readmission, suggesting code sequences alone can carry sufficient predictive signal.
• Score: 0.20388938295521575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tokenization strategies shape how models process electronic health records, yet fair comparisons of their effectiveness remain limited. We present a systematic evaluation of tokenization approaches for clinical time series modeling using transformer-based architectures, revealing task-dependent and sometimes counterintuitive findings about temporal and value feature importance. Through controlled ablations across four clinical prediction tasks on MIMIC-IV, we demonstrate that explicit time encodings provide no consistent statistically significant benefit for the evaluated downstream tasks. Value features show task-dependent importance, affecting mortality prediction but not readmission, suggesting code sequences alone can carry sufficient predictive signal. We further show that frozen pretrained code encoders dramatically outperform their trainable counterparts while requiring dramatically fewer parameters. Larger clinical encoders provide consistent improvements across tasks, benefiting from frozen embeddings that eliminate computational overhead. Our controlled evaluation enables fairer tokenization comparisons and demonstrates that simpler, parameter-efficient approaches can, in many cases, achieve strong performance, though the optimal tokenization strategy remains task-dependent.

Related papers

• Using Explainability as a Training-Time Reliability Signal for Efficient ECG Classification [14.279568423870572]
  We introduce ERTS, an explainability-based reliability training signal for efficient ECG classification.<n>ERTS uses explanation quality during training to distinguish between informative and unreliable uncertainty.<n>We evaluate ERTS across three ECG datasets and multiple backbone architectures, showing consistent improvements in macro-F1 alongside reduced effective training cost.
  arXiv  Detail & Related papers  (2026-06-10T15:55:00Z)
• Benchmarking Machine Learning Architectures for Antimicrobial Stewardship in Pediatric ICUs [1.113021697417606]
  Antimicrobial stewardship (AMS) is critical in pediatric intensive care units (PICUs)<n>Machine learning offers a promising approach for identifying patient-level opportunities for stewardship interventions from electronic health record data.<n>We present a systematic benchmarking study of AMS intervention prediction in the PICU across a public dataset and a private institutional cohort.
  arXiv  Detail & Related papers  (2026-05-21T15:26:58Z)
• CAETC: Causal Autoencoding and Treatment Conditioning for Counterfactual Estimation over Time [11.100922246022343]
  We introduce causal autoencoding and treatment conditioning (CAETC), a novel method for this problem.<n>Built on adversarial representation learning, our method leverages an autoencoding architecture to learn a partially invertible and treatment-invariant representation.<n>We conduct extensive experiments on synthetic, semi-synthetic, and real-world data to demonstrate that CAETC yields significant improvement in counterfactual estimation over existing methods.
  arXiv  Detail & Related papers  (2026-03-12T05:37:56Z)
• Tokenization Tradeoffs in Structured EHR Foundation Models [0.23453441553817042]
  Tokenization determines what information is preserved, how efficiently it is encoded, and which relationships must be learned versus precomputed.<n>Here, we pretrained a transformer on pediatric EHR data under a factorial design, varying tokenization along event encoding, time encoding, and workflow annotation.<n>Joint event encoding and positional time encoding outperformed their alternatives while requiring 39.5% and 9.6% fewer pretraining floating-point operations, respectively.
  arXiv  Detail & Related papers  (2026-03-03T22:48:39Z)
• Time-to-Event Transformer to Capture Timing Attention of Events in EHR Time Series [15.049813932448112]
  LITT is a novel Timing-Transformer architecture that enables temporary alignment of sequential events on a virtual relative timeline''<n>Its interpretability and effectiveness are validated on real-world longitudinal EHR data from 3,276 breast cancer patients.
  arXiv  Detail & Related papers  (2026-02-11T00:13:08Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• Latent Factor Point Processes for Patient Representation in Electronic Health Records [12.377029213322198]
  Most statistical methods reduce the irregular timing of EHR codes into simple counts, thereby discarding rich temporal structure.<n>We propose the latent factor point process model, which represents code occurrences as a high-dimensional point process.<n>This low-rank structure reflects the clinical reality that thousands of codes are governed by a small number of underlying disease processes.
  arXiv  Detail & Related papers  (2025-08-28T00:08:55Z)
• Early Prediction of Causes (not Effects) in Healthcare by Long-Term Clinical Time Series Forecasting [11.96384267146423]
  We propose to directly predict the causes via time series forecasting (TSF) of clinical variables. Because model training does not rely on a particular label anymore, the forecasted data can be used to predict any consensus-based label.
  arXiv  Detail & Related papers  (2024-08-07T14:52:06Z)
• Causal Contrastive Learning for Counterfactual Regression Over Time [3.3523758554338734]
  This paper introduces a unique approach to counterfactual regression over time, emphasizing long-term predictions. Distinguishing itself from existing models like Causal Transformer, our approach highlights the efficacy of employing RNNs for long-term forecasting. Our method achieves state-of-the-art counterfactual estimation results using both synthetic and real-world data.
  arXiv  Detail & Related papers  (2024-06-01T19:07:25Z)
• A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics [51.147876395589925]
  A non-stationary PGDS is proposed to allow the underlying transition matrices to evolve over time. A fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed non-stationary PGDS achieves improved predictive performance.
  arXiv  Detail & Related papers  (2024-02-26T04:39:01Z)
• PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection [65.24854366973794]
  Node-level graph anomaly detection (GAD) plays a critical role in identifying anomalous nodes from graph-structured data in domains such as medicine, social networks, and e-commerce. We introduce a simple method termed PREprocessing and Matching (PREM for short) to improve the efficiency of GAD. Our approach streamlines GAD, reducing time and memory consumption while maintaining powerful anomaly detection capabilities.
  arXiv  Detail & Related papers  (2023-10-18T02:59:57Z)
• Time Associated Meta Learning for Clinical Prediction [78.99422473394029]
  We propose a novel time associated meta learning (TAML) method to make effective predictions at multiple future time points. To address the sparsity problem after task splitting, TAML employs a temporal information sharing strategy to augment the number of positive samples. We demonstrate the effectiveness of TAML on multiple clinical datasets, where it consistently outperforms a range of strong baselines.
  arXiv  Detail & Related papers  (2023-03-05T03:54:54Z)
• Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounders [9.401989343015364]
  We study robust policy evaluation and policy optimization in the presence of sequentially-exogenous unobserved confounders.<n>We provide sample complexity bounds, insights, and show effectiveness both in simulations and on real-world longitudinal healthcare data of treating sepsis.
  arXiv  Detail & Related papers  (2023-02-01T18:40:53Z)
• On the Practicality of Deterministic Epistemic Uncertainty [106.06571981780591]
  deterministic uncertainty methods (DUMs) achieve strong performance on detecting out-of-distribution data. It remains unclear whether DUMs are well calibrated and can seamlessly scale to real-world applications.
  arXiv  Detail & Related papers  (2021-07-01T17:59:07Z)
• Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning [80.66108902283388]
  Multi-task learning methods should be used with caution for safety-critical applications, such as clinical risk prediction. Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss. We propose a novel temporal asymmetric multi-task learning model that performs knowledge transfer from certain tasks/timesteps to relevant uncertain tasks, based on feature-level uncertainty.
  arXiv  Detail & Related papers  (2020-06-23T06:01:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.