Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatment

• URL: http://arxiv.org/abs/2506.15019v1
• Date: Tue, 17 Jun 2025 23:10:51 GMT
• Title: Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatment
• Abstract summary: Effective reinforcement learning (RL) for sepsis treatment depends on learning stable, clinically meaningful state representations from irregular ICU time series.<n>This work demonstrates that Controlled Differential Equations (CDE) state representation can achieve strong RL policies when two key factors are met.
• Score: 8.692991479188032
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective reinforcement learning (RL) for sepsis treatment depends on learning stable, clinically meaningful state representations from irregular ICU time series. While previous works have explored representation learning for this task, the critical challenge of training instability in sequential representations and its detrimental impact on policy performance has been overlooked. This work demonstrates that Controlled Differential Equations (CDE) state representation can achieve strong RL policies when two key factors are met: (1) ensuring training stability through early stopping or stabilization methods, and (2) enforcing acuity-aware representations by correlation regularization with clinical scores (SOFA, SAPS-II, OASIS). Experiments on the MIMIC-III sepsis cohort reveal that stable CDE autoencoder produces representations strongly correlated with acuity scores and enables RL policies with superior performance (WIS return $> 0.9$). In contrast, unstable CDE representation leads to degraded representations and policy failure (WIS return $\sim$ 0). Visualizations of the latent space show that stable CDEs not only separate survivor and non-survivor trajectories but also reveal clear acuity score gradients, whereas unstable training fails to capture either pattern. These findings highlight practical guidelines for using CDEs to encode irregular medical time series in clinical RL, emphasizing the need for training stability in sequential representation learning.

Related papers

• Uncertainty-Guided LLM Semantic Augmentation for Heterogeneous Treatment Effect Estimation [9.269164572474216]
  Estimating heterogeneous treatment effects is central to targeted interventions, such as personalized promotions and precision medicine.<n>We focus on the conditional average treatment effect (CATE), a standard estimand for characterizing such heterogeneity.<n>We propose CURL (Causal Uncertainty-guided Representation Learning), a plug-in adapter that uses estimator uncertainty to allocate pretrained semantic capacity to locally unstable units.
  arXiv  Detail & Related papers  (2026-07-29T08:20:17Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management [4.772536418776007]
  We argue that chronic disease management is structurally a more tractable RL setting than the acute-care problems the field has primarily studied.<n>We present simulation results on synthetic state machines for hypertension and type 2 diabetes.
  arXiv  Detail & Related papers  (2026-05-10T23:43:39Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction [2.482401782570932]
  Early prediction of severe clinical deterioration can enable timely intervention and better resource allocation in high-acuity settings such as the ICU.<n>This has driven the development of machine learning models that leverage continuous streams of vital signs and other physiological signals for real-time risk prediction.<n>We propose OC-Distill, a two-stage framework that leverages multimodal supervision during training while requiring only vital signs at inference.
  arXiv  Detail & Related papers  (2026-04-18T07:01:28Z)
• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning [50.22167852149165]
  We propose EviDep, an evidential learning framework that jointly quantifies depression severity.<n>EviDep enforces strict information integrity to guarantee robust evidence synthesis.<n>It achieves state-of-the-art predictive accuracy and superior uncertainty calibration, delivering a robust fail-safe mechanism for trustworthy clinical screening.
  arXiv  Detail & Related papers  (2026-04-17T13:27:11Z)
• GAC: Stabilizing Asynchronous RL Training for LLMs via Gradient Alignment Control [16.529035487811267]
  We show that naively applying asynchrony to policy-gradient updates can induce qualitatively different training dynamics and lead to severe training instability.<n>We propose GRADIENT ALIGNMENT CONTROL, a simple dynamics-aware stabilization method that regulates asynchronous RL progress along stale-aligned directions.
  arXiv  Detail & Related papers  (2026-03-02T06:19:43Z)
• ThreatFormer-IDS: Robust Transformer Intrusion Detection with Zero-Day Generalization and Explainable Attribution [0.0]
  Intrusion detection in IoT and industrial networks requires models that can detect rare attacks at low false-positive rates while remaining reliable under evolving traffic and limited labels.<n>We propose ThreatFormer- IDS, a Transformer-based sequence modeling framework that converts flow records into time-ordered windows and learns contextual representations for robust intrusion screening.<n>On the ToN IoT benchmark with chronological evaluation, ThreatFormer-IDS achieves AUCROC 0.994, AUC-PR 0.956, and Recall@1%FPR 0.910, outperforming strong tree-based and sequence baselines.
  arXiv  Detail & Related papers  (2026-02-26T23:20:42Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• ClinNet: Evidential Ordinal Regression with Bilateral Asymmetry and Prototype Memory for Knee Osteoarthritis Grading [3.337151338735509]
  Knee osteoarthritis (KOA) grading based on radiographic images is a critical yet challenging task.<n>In this work, we propose ClinNet, a novel trustworthy framework that addresses KOA grading as evidential ordinal regression problem.
  arXiv  Detail & Related papers  (2026-01-24T05:49:41Z)
• Representation-Aware Unlearning via Activation Signatures: From Suppression to Knowledge-Signature Erasure [2.0017902634527194]
  We introduce Knowledge Immunization Framework (KIF), a representation-aware architecture that distinguishes genuine erasure from obfuscation.<n>Our approach combines dynamic suppression of subject-specific representations with parameter-efficient adaptation, enabling durable unlearning without full model retraining.
  arXiv  Detail & Related papers  (2026-01-15T16:28:14Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• Exploring Structural Degradation in Dense Representations for Self-supervised Learning [84.52554180480037]
  We observe a counterintuitive phenomenon in self-supervised learning (SSL): longer training may impair the performance of dense prediction tasks.<n>We refer to this phenomenon as Self-supervised Dense Degradation (SDD) and demonstrate its consistent presence across sixteen state-of-the-art SSL methods.<n>We introduce a Dense representation Structure Estimator (DSE) composed of a class-relevance measure and an effective dimensionality measure.
  arXiv  Detail & Related papers  (2025-10-20T08:40:16Z)
• Parameterized Diffusion Optimization enabled Autoregressive Ordinal Regression for Diabetic Retinopathy Grading [53.11883409422728]
  This work proposes a novel autoregressive ordinal regression method called AOR-DR.<n>We decompose the diabetic retinopathy grading task into a series of ordered steps by fusing the prediction of the previous steps with extracted image features.<n>We exploit the diffusion process to facilitate conditional probability modeling, enabling the direct use of continuous global image features for autoregression.
  arXiv  Detail & Related papers  (2025-07-07T13:22:35Z)
• DyCON: Dynamic Uncertainty-aware Consistency and Contrastive Learning for Semi-supervised Medical Image Segmentation [18.102129546708905]
  We present DyCON, a Dynamic Uncertainty-aware Consistency and Contrastive Learning framework.<n>UnCL enforces global consistency by dynamically weighting the contribution of each voxel to the consistency loss.<n>FeCL enhances local feature discrimination in imbalanced regions by introducing dual focal mechanisms.
  arXiv  Detail & Related papers  (2025-04-06T17:50:22Z)
• Cross-Modal Consistency Learning for Sign Language Recognition [92.44927164283641]
  Existing pre-training methods solely focus on the compact pose data.<n>We propose a Cross-modal Consistency Learning framework (CCL- SLR)<n>CCL- SLR learns from both RGB and pose modalities based on self-supervised pre-training.
  arXiv  Detail & Related papers  (2025-03-16T12:34:07Z)
• Dual Invariance Self-training for Reliable Semi-supervised Surgical Phase Recognition [5.7977777220041204]
  Semi-supervised learning, particularly with pseudo-labeling, shows promise over fully supervised methods but often lacks reliable pseudo-label assessment mechanisms.<n>We propose a novel SSL framework, Dual Invariance Self-Training (DIST), that incorporates both Temporal and Transformation Invariance.<n>Our two-step self-training process dynamically selects reliable pseudo-labels, ensuring robust pseudo-supervision.<n>Our approach mitigates the risk of noisy pseudo-labels, steering decision boundaries toward true data distribution and improving generalization to unseen data.
  arXiv  Detail & Related papers  (2025-01-29T13:07:56Z)
• An AI System for Continuous Knee Osteoarthritis Severity Grading Using Self-Supervised Anomaly Detection with Limited Data [0.30723404270319693]
  This work proposes a three stage approach for automated continuous grading of knee OA.<n>It learns a robust representation of healthy knee X-rays and grading disease severity based on its distance to the centre of normality.<n>The proposed methodology outperforms existing techniques by margins of up to 24% in terms of OA detection and the disease severity scores correlate with the Kellgren-Lawrence grading system at the same level as human expert performance.
  arXiv  Detail & Related papers  (2024-07-16T08:37:33Z)
• Towards stable training of parallel continual learning [27.774814769630453]
  Parallel Continual Learning tasks investigate the training methods for continual learning with multi-source input. Multiple tasks need to be trained simultaneously, leading to severe training instability in PCL. This paper introduces Stable Parallel Continual Learning (SPCL), a novel approach that enhances the training stability of PCL for both forward and backward propagation.
  arXiv  Detail & Related papers  (2024-07-11T06:31:04Z)
• Statistically Efficient Variance Reduction with Double Policy Estimation for Off-Policy Evaluation in Sequence-Modeled Reinforcement Learning [53.97273491846883]
  We propose DPE: an RL algorithm that blends offline sequence modeling and offline reinforcement learning with Double Policy Estimation. We validate our method in multiple tasks of OpenAI Gym with D4RL benchmarks.
  arXiv  Detail & Related papers  (2023-08-28T20:46:07Z)
• Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning [63.53407136812255]
  Offline Reinforcement Learning promises to learn effective policies from previously-collected, static datasets without the need for exploration. Existing Q-learning and actor-critic based off-policy RL algorithms fail when bootstrapping from out-of-distribution (OOD) actions or states. We propose Uncertainty Weighted Actor-Critic (UWAC), an algorithm that detects OOD state-action pairs and down-weights their contribution in the training objectives accordingly.
  arXiv  Detail & Related papers  (2021-05-17T20:16:46Z)
• Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images [49.1861463923357]
  We propose an uncertainty-guided dual-consistency learning network (UDC-Net) for semi-supervised COVID-19 lesion segmentation from CT images. Our proposed UDC-Net improves the fully supervised method by 6.3% in Dice and outperforms other competitive semi-supervised approaches by significant margins.
  arXiv  Detail & Related papers  (2021-04-07T16:23:35Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)
• Robust Pre-Training by Adversarial Contrastive Learning [120.33706897927391]
  Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness. We improve robustness-aware self-supervised pre-training by learning representations consistent under both data augmentations and adversarial perturbations.
  arXiv  Detail & Related papers  (2020-10-26T04:44:43Z)
• CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients [17.58391771585294]
  We propose a family of contrastive learning methods, CLOCS, that encourage representations across space, time, textitand patients to be similar to one another. We show that CLOCS consistently outperforms the state-of-the-art methods, BYOL and SimCLR, when performing a linear evaluation of, and fine-tuning on, downstream tasks. Our training procedure naturally generates patient-specific representations that can be used to quantify patient-similarity.
  arXiv  Detail & Related papers  (2020-05-27T09:25:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.