Multimodal Learning for Multi-Omics: A Survey

• URL: http://arxiv.org/abs/2211.16509v1
• Date: Tue, 29 Nov 2022 12:08:06 GMT
• Title: Multimodal Learning for Multi-Omics: A Survey
• Authors: Sina Tabakhi, Mohammod Naimul Islam Suvon, Pegah Ahadian, Haiping Lu
• Abstract summary: Multimodal learning for integrative multi-omics analysis can help researchers and practitioners gain deep insights into human diseases. However, several challenges are hindering the development in this area, including the availability of easily accessible open-source tools. This survey aims to provide an up-to-date overview of the data challenges, fusion approaches, datasets, and software tools from several new perspectives.
• Score: 4.15790071124993
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With advanced imaging, sequencing, and profiling technologies, multiple omics data become increasingly available and hold promises for many healthcare applications such as cancer diagnosis and treatment. Multimodal learning for integrative multi-omics analysis can help researchers and practitioners gain deep insights into human diseases and improve clinical decisions. However, several challenges are hindering the development in this area, including the availability of easily accessible open-source tools. This survey aims to provide an up-to-date overview of the data challenges, fusion approaches, datasets, and software tools from several new perspectives. We identify and investigate various omics data challenges that can help us understand the field better. We categorize fusion approaches comprehensively to cover existing methods in this area. We collect existing open-source tools to facilitate their broader utilization and development. We explore a broad range of omics data modalities and a list of accessible datasets. Finally, we summarize future directions that can potentially address existing gaps and answer the pressing need to advance multimodal learning for multi-omics data analysis.

Related papers

• Multimodal Alignment and Fusion: A Survey [7.250878248686215]
  Multimodal integration enables improved model accuracy and broader applicability. We systematically categorize and analyze existing alignment and fusion techniques. This survey focuses on applications in domains like social media analysis, medical imaging, and emotion recognition.
  arXiv  Detail & Related papers  (2024-11-26T02:10:27Z)
• A Survey of Medical Vision-and-Language Applications and Their Techniques [48.268198631277315]
  Medical vision-and-language models (MVLMs) have attracted substantial interest due to their capability to offer a natural language interface for interpreting complex medical data. Here, we provide a comprehensive overview of MVLMs and the various medical tasks to which they have been applied. We also examine the datasets used for these tasks and compare the performance of different models based on standardized evaluation metrics.
  arXiv  Detail & Related papers  (2024-11-19T03:27:05Z)
• A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications [0.7831774233149619]
  This systematic review aims to analyze and formalize current intermediate fusion methods in biomedical applications. We introduce a structured notation to enhance the understanding and application of these methods beyond the biomedical domain. Our findings are intended to support researchers, healthcare professionals, and the broader deep learning community in developing more sophisticated and insightful multimodal models.
  arXiv  Detail & Related papers  (2024-08-02T11:48:04Z)
• Automated Ensemble Multimodal Machine Learning for Healthcare [52.500923923797835]
  We introduce a multimodal framework, AutoPrognosis-M, that enables the integration of structured clinical (tabular) data and medical imaging using automated machine learning. AutoPrognosis-M incorporates 17 imaging models, including convolutional neural networks and vision transformers, and three distinct multimodal fusion strategies.
  arXiv  Detail & Related papers  (2024-07-25T17:46:38Z)
• A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data [41.8344712915454]
  Automatic radiology report generation can alleviate the workload for physicians and minimize regional disparities in medical resources. It is a challenging task, as the computational model needs to mimic physicians to obtain information from multi-modal input data. Recent works emerged to address this issue using deep learning-based methods, such as transformers, contrastive learning, and knowledge-base construction. This survey summarizes the key techniques developed in the most recent works and proposes a general workflow for deep learning-based report generation.
  arXiv  Detail & Related papers  (2024-05-21T14:37:35Z)
• Multimodal Fusion on Low-quality Data: A Comprehensive Survey [110.22752954128738]
  This paper surveys the common challenges and recent advances of multimodal fusion in the wild. We identify four main challenges that are faced by multimodal fusion on low-quality data. This new taxonomy will enable researchers to understand the state of the field and identify several potential directions.
  arXiv  Detail & Related papers  (2024-04-27T07:22:28Z)
• Review of multimodal machine learning approaches in healthcare [0.0]
  Clinicians rely on a variety of data sources to make informed decisions. Recent advances in machine learning have facilitated the more efficient incorporation of multimodal data.
  arXiv  Detail & Related papers  (2024-02-04T12:21:38Z)
• Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges [58.32937972322058]
  "Medico automatic polyp segmentation (Medico 2020)" and "MedAI: Transparency in Medical Image (MedAI 2021)" competitions. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic.
  arXiv  Detail & Related papers  (2023-07-30T16:08:45Z)
• Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework [51.44863255495668]
  Multimodal reasoning is a critical component in the pursuit of artificial intelligence systems that exhibit human-like intelligence. We present Multi-Modal Reasoning(COCO-MMR) dataset, a novel dataset that encompasses an extensive collection of open-ended questions. We propose innovative techniques, including multi-hop cross-modal attention and sentence-level contrastive learning, to enhance the image and text encoders.
  arXiv  Detail & Related papers  (2023-07-24T08:58:25Z)
• Incomplete Multimodal Learning for Complex Brain Disorders Prediction [65.95783479249745]
  We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative cohort.
  arXiv  Detail & Related papers  (2023-05-25T16:29:16Z)
• Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources [40.32772510980854]
  "patchwork learning" (PL) is a paradigm that integrates information from disparate datasets composed of different data modalities. PL allows the simultaneous utilization of complementary data sources while preserving data privacy. We present the concept of patchwork learning and its current implementations in healthcare, exploring the potential opportunities and applicable data sources.
  arXiv  Detail & Related papers  (2023-05-10T14:50:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.