Longitudinal Ensemble Integration for sequential classification with multimodal data

• URL: http://arxiv.org/abs/2411.05983v1
• Date: Fri, 08 Nov 2024 21:31:48 GMT
• Title: Longitudinal Ensemble Integration for sequential classification with multimodal data
• Authors: Aviad Susman, Repack Krishnamurthy, Richard Yan Chak Li, Mohammad Olaimat, Serdar Bozdag, Bino Varghese, Nasim Sheikh-Bahei, Gaurav Pandey,
• Abstract summary: We developed Longitudinal Ensemble Integration (LEI) for sequential classification. We evaluated LEI's performance, and compared it against existing approaches, for the early detection of dementia. LEI's design also enabled the identification of features that were consistently important across time for the effective prediction of dementia-related diagnoses.
• Score: 2.4554016712597138
• License:
• Abstract: Effectively modeling multimodal longitudinal data is a pressing need in various application areas, especially biomedicine. Despite this, few approaches exist in the literature for this problem, with most not adequately taking into account the multimodality of the data. In this study, we developed multiple configurations of a novel multimodal and longitudinal learning framework, Longitudinal Ensemble Integration (LEI), for sequential classification. We evaluated LEI's performance, and compared it against existing approaches, for the early detection of dementia, which is among the most studied multimodal sequential classification tasks. LEI outperformed these approaches due to its use of intermediate base predictions arising from the individual data modalities, which enabled their better integration over time. LEI's design also enabled the identification of features that were consistently important across time for the effective prediction of dementia-related diagnoses. Overall, our work demonstrates the potential of LEI for sequential classification from longitudinal multimodal data.

Related papers

• Supervised Multi-Modal Fission Learning [19.396207029419813]
  Learning from multimodal datasets can leverage complementary information and improve performance in prediction tasks. We propose a Multi-Modal Fission Learning model that simultaneously identifies globally joint, partially joint, and individual components.
  arXiv  Detail & Related papers  (2024-09-30T17:58:03Z)
• Multi-Epoch learning with Data Augmentation for Deep Click-Through Rate Prediction [53.88231294380083]
  We introduce a novel Multi-Epoch learning with Data Augmentation (MEDA) framework, suitable for both non-continual and continual learning scenarios. MEDA minimizes overfitting by reducing the dependency of the embedding layer on subsequent training data. Our findings confirm that pre-trained layers can adapt to new embedding spaces, enhancing performance without overfitting.
  arXiv  Detail & Related papers  (2024-06-27T04:00:15Z)
• EMERGE: Integrating RAG for Improved Multimodal EHR Predictive Modeling [22.94521527609479]
  EMERGE is a Retrieval-Augmented Generation driven framework aimed at enhancing multimodal EHR predictive modeling. Our approach extracts entities from both time-series data and clinical notes by prompting Large Language Models. The extracted knowledge is then used to generate task-relevant summaries of patients' health statuses.
  arXiv  Detail & Related papers  (2024-05-27T10:53:15Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• On the Importance of Step-wise Embeddings for Heterogeneous Clinical Time-Series [1.3285222309805063]
  Recent advances in deep learning for sequence modeling have not fully transferred to tasks handling time-series from electronic health records. In particular, in problems related to the Intensive Care Unit (ICU), the state-of-the-art remains to tackle sequence classification in a tabular manner with tree-based methods.
  arXiv  Detail & Related papers  (2023-11-15T12:18:15Z)
• Learnable Weight Initialization for Volumetric Medical Image Segmentation [66.3030435676252]
  We propose a learnable weight-based hybrid medical image segmentation approach. Our approach is easy to integrate into any hybrid model and requires no external training data. Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-15T17:55:05Z)
• Time Associated Meta Learning for Clinical Prediction [78.99422473394029]
  We propose a novel time associated meta learning (TAML) method to make effective predictions at multiple future time points. To address the sparsity problem after task splitting, TAML employs a temporal information sharing strategy to augment the number of positive samples. We demonstrate the effectiveness of TAML on multiple clinical datasets, where it consistently outperforms a range of strong baselines.
  arXiv  Detail & Related papers  (2023-03-05T03:54:54Z)
• Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework [89.8609061423685]
  We propose an information-theoretic approach to quantify the degree of redundancy, uniqueness, and synergy relating input modalities with an output task. To validate PID estimation, we conduct extensive experiments on both synthetic datasets where the PID is known and on large-scale multimodal benchmarks. We demonstrate their usefulness in (1) quantifying interactions within multimodal datasets, (2) quantifying interactions captured by multimodal models, (3) principled approaches for model selection, and (4) three real-world case studies.
  arXiv  Detail & Related papers  (2023-02-23T18:59:05Z)
• Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis [0.15776842283814416]
  We present a new technical approach of "learnable synergies" Our approach is easily scalable and naturally adapts to multimodal data inputs from clinical routine. It outperforms state-of-the-art models in clinically relevant diagnosis tasks.
  arXiv  Detail & Related papers  (2022-12-18T20:43:37Z)
• HiRID-ICU-Benchmark -- A Comprehensive Machine Learning Benchmark on High-resolution ICU Data [0.8418021941792283]
  We aim to provide a benchmark covering a large spectrum of ICU-related tasks. Using the HiRID dataset, we define multiple clinically relevant tasks developed in collaboration with clinicians. We provide an in-depth analysis of current state-of-the-art sequence modeling methods, highlighting some limitations of deep learning approaches for this type of data.
  arXiv  Detail & Related papers  (2021-11-16T15:06:42Z)
• Unpaired Multi-modal Segmentation via Knowledge Distillation [77.39798870702174]
  We propose a novel learning scheme for unpaired cross-modality image segmentation. In our method, we heavily reuse network parameters, by sharing all convolutional kernels across CT and MRI. We have extensively validated our approach on two multi-class segmentation problems.
  arXiv  Detail & Related papers  (2020-01-06T20:03:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.