A systematic review of challenges and proposed solutions in modeling multimodal data

• URL: http://arxiv.org/abs/2505.06945v2
• Date: Thu, 15 May 2025 11:38:48 GMT
• Title: A systematic review of challenges and proposed solutions in modeling multimodal data
• Authors: Maryam Farhadizadeh, Maria Weymann, Michael Blaß, Johann Kraus, Christopher Gundler, Sebastian Walter, Noah Hempen, Harald Binder, Nadine Binder,
• Abstract summary: Multimodal data modeling has emerged as a powerful approach in clinical research.<n>This systematic review synthesizes findings from 69 studies to identify common obstacles.<n>We highlight recent methodological advances, such as transfer learning, generative models, attention mechanisms, and neural architecture search that offer promising solutions.
• Score: 0.9674145073701153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records. Despite its potential to improve diagnostic accuracy and support personalized care, modeling such heterogeneous data presents significant technical challenges. This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques. We highlight recent methodological advances, such as transfer learning, generative models, attention mechanisms, and neural architecture search that offer promising solutions. By mapping current trends and innovations, this review provides a comprehensive overview of the field and offers practical insights to guide future research and development in multimodal modeling for medical applications.

Related papers

• Anomaly Detection and Generation with Diffusion Models: A Survey [51.61574868316922]
  Anomaly detection (AD) plays a pivotal role across diverse domains, including cybersecurity, finance, healthcare, and industrial manufacturing.<n>Recent advancements in deep learning, specifically diffusion models (DMs), have sparked significant interest.<n>This survey aims to guide researchers and practitioners in leveraging DMs for innovative AD solutions across diverse applications.
  arXiv  Detail & Related papers  (2025-06-11T03:29:18Z)
• Generative Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges [44.77761945679817]
  Generative modeling has emerged as a promising direction in computational pathology.<n>Generative models offer capabilities such as data-efficient learning, synthetic data augmentation, and multimodal representation.<n>This review provides a comprehensive synthesis of recent progress in the field.
  arXiv  Detail & Related papers  (2025-05-16T08:44:50Z)
• Recent Advances in Medical Imaging Segmentation: A Survey [7.72661433458686]
  Generative AI, Few-Shot Learning, Foundation Models, and Universal Models offered promising solutions to longstanding challenges.<n>We discuss inherent limitations, unresolved issues, and future research directions aimed at enhancing the practicality and accessibility of segmentation models in medical imaging.
  arXiv  Detail & Related papers  (2025-05-14T10:48:37Z)
• Biomedical Foundation Model: A Survey [84.26268124754792]
  Foundation models are large-scale pre-trained models that learn from extensive unlabeled datasets.<n>These models can be adapted to various applications such as question answering and visual understanding.<n>This survey explores the potential of foundation models across diverse domains within biomedical fields.
  arXiv  Detail & Related papers  (2025-03-03T22:42:00Z)
• From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine [40.23383597339471]
  multimodal AI is capable of integrating diverse data modalities, including imaging, text, and structured data, within a single model.<n>This scoping review explores the evolution of multimodal AI, highlighting its methods, applications, datasets, and evaluation in clinical settings.<n>Our findings underscore a shift from unimodal to multimodal approaches, driving innovations in diagnostic support, medical report generation, drug discovery, and conversational AI.
  arXiv  Detail & Related papers  (2025-02-13T11:57:51Z)
• A Survey on Diffusion Models for Anomaly Detection [41.22298168457618]
  Diffusion models (DMs) have emerged as a powerful class of generative AI models.<n>DMAD offers promising solutions for identifying deviations in increasingly complex and high-dimensional data.
  arXiv  Detail & Related papers  (2025-01-20T12:06:54Z)
• Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applications [2.3754862363513523]
  This review examines the landscape of deep learning-based multimodal AI applications across the medical domain. multimodal AI models consistently outperform their unimodal counterparts, with an average improvement of 6.2 percentage points in AUC. We identify key factors driving multimodal AI development and propose recommendations to accelerate the field's maturation.
  arXiv  Detail & Related papers  (2024-11-06T09:18:05Z)
• Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospects [2.1070612998322438]
  The paper explores the transformative potential of multimodal models for clinical predictions. Despite advancements, challenges such as data biases and the scarcity of "big data" in many biomedical domains persist.
  arXiv  Detail & Related papers  (2023-11-04T05:42:51Z)
• Incomplete Multimodal Learning for Complex Brain Disorders Prediction [65.95783479249745]
  We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative cohort.
  arXiv  Detail & Related papers  (2023-05-25T16:29:16Z)
• Domain Shift in Computer Vision models for MRI data analysis: An Overview [64.69150970967524]
  Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use. Poor transferability of themodels to data from different sources or acquisition domains is one of the reasons for that.
  arXiv  Detail & Related papers  (2020-10-14T16:34:21Z)
• Machine Learning in Nano-Scale Biomedical Engineering [77.75587007080894]
  We review the existing research regarding the use of machine learning in nano-scale biomedical engineering. The main challenges that can be formulated as ML problems are classified into the three main categories. For each of the presented methodologies, special emphasis is given to its principles, applications, and limitations.
  arXiv  Detail & Related papers  (2020-08-05T15:45:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.