LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts

• URL: http://arxiv.org/abs/2606.09907v1
• Date: Sat, 06 Jun 2026 07:50:41 GMT
• Title: LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
• Abstract summary: Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records.<n>It faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, and ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time.<n>We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges.
• Score: 2.537429062537451
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records. However, it faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time. Existing methods address these challenges in isolation: missing-modality frameworks treat each visit as an independent static snapshot and discard temporal context, while longitudinal models often assume complete modality availability and degrade under systematic modality incompleteness. We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges. LongMoE combines a context-aware imputation module with an attentional tokenization module that captures frequency-domain temporal patterns across irregular visit sequences, a trajectory-aware encoder for modeling disease progression, and context-conditioned Sparse MoE routing for patient-specific expert selection. Experiments on ADNI, OASIS-3, and MIMIC-IV show that LongMoE improves robustness under missing or weak contemporaneous modalities and remains competitive in full-modality settings, establishing a strong foundation for longitudinally-aware multimodal clinical learning.

Related papers

• Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning [16.941311033409626]
  We introduce Latent-Attention Masked Autoencoders (LAMAE), a multimodal, structure-aware masked autoencoder.<n>LAMAE exchanges information directly in the latent space through a shared latent-attention module operating over a study-view-entity hierarchy.<n>LAMAE outperforms modality-specific pretraining and strong contrastive and vision-language baselines across multimodal hospital-stay tasks.
  arXiv  Detail & Related papers  (2026-09-10T17:49:44Z)
• DeMMO: Longitudinal and Cross-Disease Modelling of Digital Mobility Outcomes via Multi-Task Learning [6.318710451121123]
  Digital mobility outcomes (DMOs) derived from wearable sensors offer a promising means of monitoring disease progression.<n>Existing DMO studies have typically focused on either a single disease or a single visit.<n>We propose DeMMO, an interpretable framework for longitudinal, multi-disease, and multi-outcome learning.
  arXiv  Detail & Related papers  (2026-08-25T19:07:32Z)
• AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis [57.942670844512016]
  We introduce AdaSurvMamba as a novel adaptive framework for multimodal survival analysis.<n>The framework features a Dual-Scale Importance-Aware Reconstruction (DSIR) module to dynamically modulate cross-modal interaction strength.<n>Experiments across five TCGA cohorts demonstrate consistent gains over existing methods.
  arXiv  Detail & Related papers  (2026-06-28T11:36:31Z)
• MedMamba: Multi-View State Space Models with Adaptive Graph Learning for Medical Time Series Classification [60.41879345743941]
  We propose MedMamba, an end-to-end architecture that integrates state space models with domain-specific inductive biases.<n>MedMamba achieves state-of-the-art performance while maintaining linear computational complexity.
  arXiv  Detail & Related papers  (2026-05-24T09:23:39Z)
• Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling [24.896312231457653]
  We develop a framework to profile and handle missing modalities while addressing the canonical desideratum of safe, transparent clinical AI.<n>We show that autoregressive sequence modeling with transformer-based architectures outperforms baselines on the MIMIC-IV and eICU fine-tuning benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T18:55:56Z)
• Learning Patient-Specific Disease Dynamics with Latent Flow Matching for Longitudinal Imaging Generation [17.33607122354623]
  Understanding disease progression is a central clinical challenge with implications for early diagnosis and personalized treatment.<n>We propose to treat the disease dynamic as a velocity field and leverage Flow Matching (FM) to align the temporal evolution of patient data.<n>We present $$-LFM, a framework for modeling patient-specific latent progression with flow matching.
  arXiv  Detail & Related papers  (2025-12-09T23:13:54Z)
• TARDis: Time Attenuated Representation Disentanglement for Incomplete Multi-Modal Tumor Segmentation and Classification [10.329406702659123]
  Tumor segmentation and diagnosis in contrast-enhanced Computed Tomography (CT) rely heavily on the physiological dynamics of contrast agents.<n>Existing deep learning approaches typically treat missing phases as absent independent channels, ignoring the inherent temporal continuity of hemodynamics.<n>We propose Time Attenuated Representation Disentanglement (TARDis), a novel physics-aware framework that redefines missing modalities as missing sample points on a continuous Time-Attenuation Curve.
  arXiv  Detail & Related papers  (2025-12-04T08:44:50Z)
• MIRROR: Multi-Modal Pathological Self-Supervised Representation Learning via Modality Alignment and Retention [57.044719143401664]
  Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease.<n>We present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention.<n>Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR's superior performance.
  arXiv  Detail & Related papers  (2025-03-01T07:02:30Z)
• MVICAD2: Multi-View Independent Component Analysis with Delays and Dilations [61.59658203704757]
  We propose Multi-View Independent Component Analysis with Delays and Dilations (MVICAD2), which allows sources to differ across subjects in both temporal delays and dilations.<n>We present a model with identifiable sources, derive an approximation of its likelihood in closed form, and use regularization and optimization techniques to enhance performance.
  arXiv  Detail & Related papers  (2025-01-13T15:47:02Z)
• HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generation [89.3260120072177]
  We propose a novel Historical-Constrained Large Language Models (HC-LLM) framework for Radiology report generation.<n>Our approach extracts both time-shared and time-specific features from longitudinal chest X-rays and diagnostic reports to capture disease progression.<n> Notably, our approach performs well even without historical data during testing and can be easily adapted to other multimodal large models.
  arXiv  Detail & Related papers  (2024-12-15T06:04:16Z)
• CTPD: Cross-Modal Temporal Pattern Discovery for Enhanced Multimodal Electronic Health Records Analysis [50.56875995511431]
  We introduce a Cross-Modal Temporal Pattern Discovery (CTPD) framework, designed to efficiently extract meaningful cross-modal temporal patterns from multimodal EHR data.<n>Our approach introduces shared initial temporal pattern representations which are refined using slot attention to generate temporal semantic embeddings.
  arXiv  Detail & Related papers  (2024-11-01T15:54:07Z)
• Global Contrastive Training for Multimodal Electronic Health Records with Language Supervision [1.6245786035158123]
  This paper introduces a novel multimodal contrastive learning framework, specifically focusing on medical time series and clinical notes. The framework integrates temporal cross-attention transformers with a dynamic embedding and tokenization scheme for learning multimodal feature representations. Experiments with a real-world EHR dataset demonstrated that our framework outperformed state-of-the-art approaches on the exemplar task of predicting the occurrence of nine postoperative complications.
  arXiv  Detail & Related papers  (2024-04-10T04:19:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.