Multi-Scale and Multi-Modal Contrastive Learning Network for Biomedical Time Series

• URL: http://arxiv.org/abs/2312.03796v1
• Date: Wed, 6 Dec 2023 14:33:50 GMT
• Title: Multi-Scale and Multi-Modal Contrastive Learning Network for Biomedical Time Series
• Abstract summary: This paper presents a multi-scale and multi-modal biomedical time series representation learning (MBSL) network with contrastive learning to migrate variations. Experiments against four bio-medical applications show that MBSL outperforms state-of-the-art models by 33.9% mean average errors (MAE) in respiration rate, by 13.8% MAE in exercise heart rate, by 1.41% accuracy in human activity recognition, and by 1.14% F1-score in obstructive sleep apnea-hypopnea syndrome.
• Score: 3.769171969383224
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal biomedical time series (MBTS) data offers a holistic view of the physiological state, holding significant importance in various bio-medical applications. Owing to inherent noise and distribution gaps across different modalities, MBTS can be complex to model. Various deep learning models have been developed to learn representations of MBTS but still fall short in robustness due to the ignorance of modal-to-modal variations. This paper presents a multi-scale and multi-modal biomedical time series representation learning (MBSL) network with contrastive learning to migrate these variations. Firstly, MBTS is grouped based on inter-modal distances, then each group with minimum intra-modal variations can be effectively modeled by individual encoders. Besides, to enhance the multi-scale feature extraction (encoder), various patch lengths and mask ratios are designed to generate tokens with semantic information at different scales and diverse contextual perspectives respectively. Finally, cross-modal contrastive learning is proposed to maximize consistency among inter-modal groups, maintaining useful information and eliminating noises. Experiments against four bio-medical applications show that MBSL outperforms state-of-the-art models by 33.9% mean average errors (MAE) in respiration rate, by 13.8% MAE in exercise heart rate, by 1.41% accuracy in human activity recognition, and by 1.14% F1-score in obstructive sleep apnea-hypopnea syndrome.

Related papers

• MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models [83.50100003741628]
  Vision and language models (VLMs) hold immense promise to transform biomedical imaging.<n>We introduce the Massive Multimodal Biomedical Understanding (MMBU) benchmark.<n>It is the largest vision and language benchmark to date, covering 35 submodalities with rich structured metadata.
  arXiv  Detail & Related papers  (2026-06-04T20:24:47Z)
• Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification [59.24009931000134]
  Multi-View Synergistic Learning (MVSL) is a unified framework that addresses adaptation paradigms, representation granularity, and disease semantic relationships.<n>MVSL decouples the adaptation of visual and textual encoders to respect their distinct representational characteristics.<n>It further introduces multi-granularity contrastive learning to explicitly model both global image semantics and localized lesion-level evidence.<n>MVSL consistently outperforms state-of-the-art methods in few-shot and zero-shot classification settings.
  arXiv  Detail & Related papers  (2026-04-27T02:41:27Z)
• BioVLM: Routing Prompts, Not Parameters, for Cross-Modality Generalization in Biomedical VLMs [38.20973682908207]
  BioVLM is a prompt-learning framework that improves cross-domain generalization without extensive backbone fine-tuning.<n>On 11 MedMNIST+ 2D datasets, BioVLM achieves new state of the art across three distinct generalization settings.
  arXiv  Detail & Related papers  (2026-04-19T21:54:53Z)
• M-IDoL: Information Decomposition for Modality-Specific and Diverse Representation Learning in Medical Foundation Model [53.96788246923603]
  We propose M-IDoL, a self-supervised underlinetextitMFM that introduces Information Decomposition for multimodal representation Learning.<n>By pre-training on 1.15 million medical images, M-IDoL i) delivers superior generalization across 21 downstream clinical tasks.
  arXiv  Detail & Related papers  (2026-04-10T04:06:11Z)
• MuSACo: Multimodal Subject-Specific Selection and Adaptation for Expression Recognition with Co-Training [52.99217736494484]
  We introduce MuSACo, a multi-modal subject-specific selection and adaptation method for personalized expression recognition.<n>This makes MuSACo relevant for affective computing applications in digital health, such as patient-specific assessment for stress or pain.<n>Our experimental results on challenging multimodal ER datasets: BioVid and StressID, show that MuSACo can outperform UDA (blending) and state-of-the-art MSDA methods.
  arXiv  Detail & Related papers  (2025-08-17T23:08:21Z)
• FoundBioNet: A Foundation-Based Model for IDH Genotyping of Glioma from Multi-Parametric MRI [1.4249472316161877]
  We propose a Foundation-based Biomarker Network (FoundBioNet) to noninvasively predict IDH mutation status from multi-parametric MRI.<n>Our model was trained and validated on a diverse, multi-center cohort of 1705 glioma patients from six public datasets.<n>Our model achieved AUCs of 90.58%, 88.08%, 65.41%, and 80.31% on independent test sets from EGD, TCGA, Ivy GAP, RHUH, and UPenn.
  arXiv  Detail & Related papers  (2025-08-09T00:08:10Z)
• Masked Omics Modeling for Multimodal Representation Learning across Histopathology and Molecular Profiles [0.0]
  Self-supervised learning has driven major advances in computational pathology.<n>However, histopathology alone often falls short for molecular characterization and understanding clinical outcomes.<n>We introduce MORPHEUS, a unified transformer-based pre-training framework that encodes both histopathology and multi-omics data into a shared latent space.
  arXiv  Detail & Related papers  (2025-08-01T15:29:26Z)
• Sparseformer: a Transferable Transformer with Multi-granularity Token Sparsification for Medical Time Series Classification [25.47662257105448]
  We introduce Sparseformer, a transformer specifically designed for MedTS classification. We propose a sparse token-based dual-attention mechanism that enables global modeling and token compression. Our model outperforms 12 baselines across seven medical datasets under supervised learning.
  arXiv  Detail & Related papers  (2025-03-19T13:22:42Z)
• Towards a Multimodal MRI-Based Foundation Model for Multi-Level Feature Exploration in Segmentation, Molecular Subtyping, and Grading of Glioma [0.2796197251957244]
  Multi-Task S-UNETR (MTSUNET) model is a novel foundation-based framework built on the BrainSegFounder model. It simultaneously performs glioma segmentation, histological subtyping and neuroimaging subtyping. It shows significant potential for advancing noninvasive, personalized glioma management by improving predictive accuracy and interpretability.
  arXiv  Detail & Related papers  (2025-03-10T01:27:09Z)
• MIND: Modality-Informed Knowledge Distillation Framework for Multimodal Clinical Prediction Tasks [50.98856172702256]
  We propose the Modality-INformed knowledge Distillation (MIND) framework, a multimodal model compression approach. MIND transfers knowledge from ensembles of pre-trained deep neural networks of varying sizes into a smaller multimodal student. We evaluate MIND on binary and multilabel clinical prediction tasks using time series data and chest X-ray images.
  arXiv  Detail & Related papers  (2025-02-03T08:50:00Z)
• Transfer Learning with Active Sampling for Rapid Training and Calibration in BCI-P300 Across Health States and Multi-centre Data [0.13124513975412253]
  Machine learning and deep learning advancements have boosted Brain-Computer Interface (BCI) performance. Their wide-scale applicability is limited due to factors like individual health, hardware variations, and cultural differences affecting neural data. We propose P300 wave detection in BCIs employing a convolutional neural network fitted with adaptive transfer learning.
  arXiv  Detail & Related papers  (2024-12-14T14:20:21Z)
• UNICORN: A Deep Learning Model for Integrating Multi-Stain Data in Histopathology [2.9389205138207277]
  UNICORN is a multi-modal transformer capable of processing multi-stain histopathology for atherosclerosis severity class prediction. The architecture comprises a two-stage, end-to-end trainable model with specialized modules utilizing transformer self-attention blocks. UNICORN achieved a classification accuracy of 0.67, outperforming other state-of-the-art models.
  arXiv  Detail & Related papers  (2024-09-26T12:13:52Z)
• Multi-omics data integration for early diagnosis of hepatocellular carcinoma (HCC) using machine learning [8.700808005009806]
  We compare the performance of ensemble machine learning algorithms capable of late integration of multi-class data from different modalities. Two boosted methods, PB-MVBoost and Adaboost with a soft vote were the overall best performing models.
  arXiv  Detail & Related papers  (2024-09-20T09:38:02Z)
• Enhanced Self-supervised Learning for Multi-modality MRI Segmentation and Classification: A Novel Approach Avoiding Model Collapse [6.3467517115551875]
  Multi-modality magnetic resonance imaging (MRI) can provide complementary information for computer-aided diagnosis. Traditional deep learning algorithms are suitable for identifying specific anatomical structures segmenting lesions and classifying diseases with magnetic resonance images. Self-supervised learning (SSL) can effectively learn feature representations from unlabeled data by pre-training and is demonstrated to be effective in natural image analysis. Most SSL methods ignore the similarity of multi-modality MRI, leading to model collapse. We establish and validate a multi-modality MRI masked autoencoder consisting of hybrid mask pattern (HMP) and pyramid barlow twin (PBT
  arXiv  Detail & Related papers  (2024-07-15T01:11:30Z)
• Interpretable cancer cell detection with phonon microscopy using multi-task conditional neural networks for inter-batch calibration [39.759100498329275]
  We present a conditional neural network framework to simultaneously achieve inter-batch calibration. We validate our approach by training and validating on different experimental batches. We extend our model to reconstruct denoised signals, enabling physical interpretation of salient features indicating disease state.
  arXiv  Detail & Related papers  (2024-03-26T12:20:10Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs [48.376109878173956]
  We present PMC-15M, a novel dataset that is two orders of magnitude larger than existing biomedical multimodal datasets. PMC-15M contains 15 million biomedical image-text pairs collected from 4.4 million scientific articles. Based on PMC-15M, we have pretrained BiomedCLIP, a multimodal foundation model, with domain-specific adaptations tailored to biomedical vision-language processing.
  arXiv  Detail & Related papers  (2023-03-02T02:20:04Z)
• Superficial White Matter Analysis: An Efficient Point-cloud-based Deep Learning Framework with Supervised Contrastive Learning for Consistent Tractography Parcellation across Populations and dMRI Acquisitions [68.41088365582831]
  White matter parcellation classifies tractography streamlines into clusters or anatomically meaningful tracts. Most parcellation methods focus on the deep white matter (DWM), whereas fewer methods address the superficial white matter (SWM) due to its complexity. We propose a novel two-stage deep-learning-based framework, Superficial White Matter Analysis (SupWMA), that performs an efficient parcellation of 198 SWM clusters from whole-brain tractography.
  arXiv  Detail & Related papers  (2022-07-18T23:07:53Z)
• Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction [125.18248926508045]
  We propose Channel-Exchanging-Network (CEN) which is self-adaptive, parameter-free, and more importantly, applicable for both multimodal fusion and multitask learning. CEN dynamically exchanges channels betweenworks of different modalities. For the application of dense image prediction, the validity of CEN is tested by four different scenarios.
  arXiv  Detail & Related papers  (2021-12-04T05:47:54Z)
• Medulloblastoma Tumor Classification using Deep Transfer Learning with Multi-Scale EfficientNets [63.62764375279861]
  We propose an end-to-end MB tumor classification and explore transfer learning with various input sizes and matching network dimensions. Using a data set with 161 cases, we demonstrate that pre-trained EfficientNets with larger input resolutions lead to significant performance improvements.
  arXiv  Detail & Related papers  (2021-09-10T13:07:11Z)
• MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data [75.73881040581767]
  We propose a novel multi-site network (MS-Net) for improving prostate segmentation by learning robust representations. Our MS-Net improves the performance across all datasets consistently, and outperforms state-of-the-art methods for multi-site learning.
  arXiv  Detail & Related papers  (2020-02-09T14:11:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.