Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding

• URL: http://arxiv.org/abs/2602.15159v1
• Date: Mon, 16 Feb 2026 19:58:44 GMT
• Title: Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding
• Abstract summary: We propose the Augmented-Intrinsic Dual-Masked Autoencoder (AID-MAE), which learns directly from incomplete time series.<n>AID-MAE processes only the unmasked subset of tokens and consistently outperforms strong baselines.
• Score: 3.6883644193065117
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning from electronic health records (EHRs) time series is challenging due to irregular sam- pling, heterogeneous missingness, and the resulting sparsity of observations. Prior self-supervised meth- ods either impute before learning, represent missingness through a dedicated input signal, or optimize solely for imputation, reducing their capacity to efficiently learn representations that support clinical downstream tasks. We propose the Augmented-Intrinsic Dual-Masked Autoencoder (AID-MAE), which learns directly from incomplete time series by applying an intrinsic missing mask to represent naturally missing values and an augmented mask that hides a subset of observed values for reconstruction during training. AID-MAE processes only the unmasked subset of tokens and consistently outperforms strong baselines, including XGBoost and DuETT, across multiple clinical tasks on two datasets. In addition, the learned embeddings naturally stratify patient cohorts in the representation space.

Related papers

• Neonatal Hypoxic-ischaemic Encephalopathy Classification from the EEG and HRV Signals Using a Conformer based Masked Autoencoder [1.2599533416395767]
  We propose a novel self-supervised learning framework for representation learning from unlabelled electroencephalography (EEG) and heart rate variability (HRV) signals.<n>By integrating convolutional operations with Transformer-based self-attention, MAEConformer effectively captures both local temporal patterns and long-range contextual dependencies.<n> Experimental results demonstrate that the learned representations provide strong transferability and data efficiency.
  arXiv  Detail & Related papers  (2026-07-26T09:11:03Z)
• Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis [93.7120829382642]
  Nemotron-Labs-Diffusion-Image is a state-of-the-art masked discrete diffusion model for text-to-image synthesis.<n>Standard MDMs lack self-correcting capability because discrete tokens cannot be modified once they are unmasked.<n>Nemotron-Labs-Diffusion-Image incorporates a token-editing mechanism that enables the model to dynamically revise already-unmasked tokens during inference.
  arXiv  Detail & Related papers  (2026-06-29T05:48:41Z)
• DARE-EEG: A Foundation Model for Mining Dual-Aligned Representation of EEG [25.4837936625773]
  We propose DARE-EEG, a self-supervised foundation model that enforces the mask-invariance property through dual-aligned representation learning.<n>Specifically, we introduce mask alignment that constrains representations from multiple masked views of the same EEG sample.<n>We also propose conv-linear-probing, a parameter-efficient strategy that adapts pre-trained representations to heterogeneous electrode configurations.
  arXiv  Detail & Related papers  (2026-05-18T12:18:43Z)
• Learning Fingerprints for Medical Time Series with Redundancy-Constrained Information Maximization [11.233142317726971]
  We propose a novel framework that compresses a variable-length MedTS into a fixed-size set of $k$ latent Fingerprint Tokens.<n>Our architecture employs a cross-attention bottleneck to generate these tokens and is trained with a dual-objective function.<n>This approach produces a low-dimensional, interpretable, and sample-efficient representation, where each token is encouraged to capture an independent factor of variation.
  arXiv  Detail & Related papers  (2026-04-30T18:33:40Z)
• RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models [12.004161606345084]
  We introduce RePrompT, a time-aware framework that integrates structured EHR encoders through prompt tuning.<n> Experiments on MIMIC-III and MIMIC-IV demonstrate that RePrompT consistently outperforms both EHR-based and LLM-based baselines.
  arXiv  Detail & Related papers  (2026-04-20T02:20:13Z)
• Digital FAST: An AI-Driven Multimodal Framework for Rapid and Early Stroke Screening [0.7136933021609076]
  This study presents a fast, non-invasive multimodal deep learning framework for automatic binary stroke screening based on data collected during the F.A.S.T. assessment.<n>The proposed approach integrates complementary information from facial expressions, speech signals, and upper-body movements to enhance diagnostic robustness.
  arXiv  Detail & Related papers  (2026-01-17T03:35:39Z)
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation [0.9342663938194833]
  MambaX-Net is a novel semi-supervised, dual-scan 3D segmentation architecture.<n>It computes the segmentation for time point t by leveraging the MRI and the corresponding segmentation mask from the previous time point.<n>MambaX-Net was evaluated on a longitudinal AS dataset, and results showed that it significantly outperforms state-of-the-art U-Net and Transformer-based models.
  arXiv  Detail & Related papers  (2025-10-20T13:32:42Z)
• Self-Supervised Cross-Encoder for Neurodegenerative Disease Diagnosis [6.226851122403944]
  We propose a novel self-supervised cross-encoder framework that leverages the temporal continuity in longitudinal MRI scans for supervision.<n>This framework disentangles learned representations into two components: a static representation, constrained by contrastive learning, which captures stable anatomical features; and a dynamic representation, guided by input-gradient regularization, which reflects temporal changes.<n> Experimental results on the Alzheimer's Disease Neuroimaging Initiative dataset demonstrate that our method achieves superior classification accuracy and improved interpretability.
  arXiv  Detail & Related papers  (2025-09-09T11:52:24Z)
• LSM-2: Learning from Incomplete Wearable Sensor Data [65.58595667477505]
  This paper introduces the second generation of Large Sensor Model (LSM-2) with Adaptive and Inherited Masking (AIM)<n>AIM learns robust representations directly from incomplete data without requiring explicit imputation.<n>Our LSM-2 with AIM achieves the best performance across a diverse range of tasks, including classification, regression and generative modeling.
  arXiv  Detail & Related papers  (2025-06-05T17:57:11Z)
• TokenUnify: Scaling Up Autoregressive Pretraining for Neuron Segmentation [65.65530016765615]
  We propose a hierarchical predictive coding framework that captures multi-scale dependencies through three complementary learning objectives.<n> TokenUnify integrates random token prediction, next-token prediction, and next-all token prediction to create a comprehensive representational space.<n>We also introduce a large-scale EM dataset with 1.2 billion annotated voxels, offering ideal long-sequence visual data with spatial continuity.
  arXiv  Detail & Related papers  (2024-05-27T05:45:51Z)
• A Masked Semi-Supervised Learning Approach for Otago Micro Labels Recognition [1.0663633381202409]
  Otago Exercise Program serves as a vital rehabilitation initiative for older adults, aiming to enhance their strength and balance, and consequently prevent falls. Existing Human Activity Recognition systems focus on the duration of macro activities. This study presents a novel semi-supervised machine learning approach aimed at bridging this gap in recognizing the micro activities of OEP.
  arXiv  Detail & Related papers  (2024-05-21T12:00:01Z)
• Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Records [1.6609516435725236]
  We introduce a dynamic embedding and tokenization framework for precise representation of multimodal clinical time series. Our framework outperformed baseline approaches on the task of predicting the occurrence of nine postoperative complications.
  arXiv  Detail & Related papers  (2024-03-06T19:46:44Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models [69.9178140563928]
  Colonoscopy analysis is essential for assisting clinical diagnosis and treatment. The scarcity of annotated data limits the effectiveness and generalization of existing methods. We propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks.
  arXiv  Detail & Related papers  (2023-09-03T07:55:46Z)
• SAITS: Self-Attention-based Imputation for Time Series [6.321652307514677]
  SAITS is a novel method based on the self-attention mechanism for missing value imputation in time series. It learns missing values from a weighted combination of two diagonally-masked self-attention blocks. Tests show SAITS outperforms state-of-the-art methods on the time-series imputation task efficiently.
  arXiv  Detail & Related papers  (2022-02-17T08:40:42Z)
• Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model [93.9943278892735]
  Key problem in protein sequence representation learning is to capture the co-evolutionary information reflected by the inter-residue co-variation in the sequences. We propose a novel method to capture this information directly by pre-training via a dedicated language model, i.e., Pairwise Masked Language Model (PMLM) Our result shows that the proposed method can effectively capture the interresidue correlations and improves the performance of contact prediction by up to 9% compared to the baseline.
  arXiv  Detail & Related papers  (2021-10-29T04:01:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.