Hierarchical Contrastive Learning for Multimodal Data

• URL: http://arxiv.org/abs/2604.05462v1
• Date: Tue, 07 Apr 2026 05:53:04 GMT
• Title: Hierarchical Contrastive Learning for Multimodal Data
• Abstract summary: Hierarchical Contrastive Learning is a framework that learns globally shared, partially shared, and modality-specific representations.<n>On multimodal electronic health records, HCL yields more informative representations and consistently improves predictive performance.
• Score: 3.1788482285024107
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal representation learning is commonly built on a shared-private decomposition, treating latent information as either common to all modalities or specific to one. This binary view is often inadequate: many factors are shared by only subsets of modalities, and ignoring such partial sharing can over-align unrelated signals and obscure complementary information. We propose Hierarchical Contrastive Learning (HCL), a framework that learns globally shared, partially shared, and modality-specific representations within a unified model. HCL combines a hierarchical latent-variable formulation with structural sparsity and a structure-aware contrastive objective that aligns only modalities that genuinely share a latent factor. Under uncorrelated latent variables, we prove identifiability of the hierarchical decomposition, establish recovery guarantees for the loading matrices, and derive parameter estimation and excess-risk bounds for downstream prediction. Simulations show accurate recovery of hierarchical structure and effective selection of task-relevant components. On multimodal electronic health records, HCL yields more informative representations and consistently improves predictive performance.

Related papers

• Beyond Independence: Learning Correlated Views for Variational Incomplete Multi-View Clustering [15.204251218204542]
  Incomplete multi-view clustering (IMVC) aims to uncover shared cluster structures from data with partially observed views.<n>We propose a variational framework that explicitly goes beyond this assumption by introducing a learnable cross-view correlation structure.<n>Our method consistently outperforms state-of-the-art approaches across diverse missing-view settings.
  arXiv  Detail & Related papers  (2026-08-16T14:16:24Z)
• SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering [51.01038967474103]
  Prototype-based Incomplete Multi-view Clustering has attracted increasing attention by exploiting prototypes as semantic anchors for missing-view imputation.<n>We propose a novel framework termed Structure-aware PrOtotype disentanglement foR incomplete multi-view clusTering (SPORT)<n>SPORT explicitly disentangles shared and view-specific components of prototypes while preserving cluster-level relational structures.
  arXiv  Detail & Related papers  (2026-07-11T17:40:22Z)
• Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery [52.29866804948133]
  We propose CAGI (Cluster-Aware Generative Imputation), a framework that reformulates clustering and imputation as a mutually reinforcing co-optimization process.<n>Experiments on 14 benchmark datasets with 15 representative baselines demonstrate the superiority of CAGI.
  arXiv  Detail & Related papers  (2026-07-08T02:50:02Z)
• RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities [34.35941675436447]
  RePercENT is a self-supervised framework designed to unlock scalable pairwise disentanglement beyond two modalities.<n>Our approach operates directly on pre-extracted embeddings, eliminating the need for extensive joint pre-training.<n>Across diverse modalities and tasks, RePercENT successfully recovers disentangled components while maintaining competitive performance.
  arXiv  Detail & Related papers  (2026-06-03T17:10:39Z)
• Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing [4.400865943835067]
  Causal representation learning seeks to uncover meaningful latent variables and their corresponding causal structure from observational data.<n>Proving identifiability in CRL is intrinsically difficult, and we address in this work an even more challenging setting: multimodality.<n>We consider multimodal observed data with a latent partially shared structure. Each modality is generated, through non linear mixing functions, from a specific subset of causal latent variables.<n>Our identifiability results, furthermore, apply to the undercomplete scenario where we have, for each modality, more observed than latent variables.
  arXiv  Detail & Related papers  (2026-05-18T21:34:29Z)
• Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection [5.097809301149341]
  In real-world deployments, some modalities may be unavailable due to sensor failures, acquisition issues, or challenging atmospheric conditions.<n>Existing multimodal segmentation models typically address missing modalities by learning a shared representation across inputs.<n>We tackle this limitation with CBC-SLP, a multimodal semantic segmentation model designed to preserve both modality-invariant and modality-specific information.
  arXiv  Detail & Related papers  (2026-04-17T09:05:22Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• Causality-Driven Disentangled Representation Learning in Multiplex Graphs [13.627536649679575]
  We introduce a causal inference-based framework that disentangles common and private components in a self-supervised manner.<n>Experiments on synthetic and real-world datasets demonstrate consistent improvements over existing baselines.
  arXiv  Detail & Related papers  (2026-03-25T09:10:28Z)
• Orthogonalized Multimodal Contrastive Learning with Asymmetric Masking for Structured Representations [4.67724003380452]
  Multimodal learning seeks to integrate information from heterogeneous sources, where signals may be shared across modalities, specific to individual modalities, or emerge only through their interaction.<n>While self-supervised multimodal contrastive learning has achieved remarkable progress, most existing methods predominantly capture redundant cross-modal signals, often neglecting modality-specific (unique) and interaction-driven (synergistic) information.<n>Recent extensions broaden this perspective, yet they either fail to explicitly model synergistic interactions or learn different information components in an entangled manner, leading to incomplete representations and potential information leakage.<n>We introduce textbfCOrAL, a principled framework
  arXiv  Detail & Related papers  (2026-02-16T18:06:53Z)
• Group Contrastive Learning for Weakly Paired Multimodal Data [34.76498775412033]
  GROOVE is a semi-supervised multi-modal representation learning approach for high-content perturbation data.<n>GroupCLIP is a novel group-level contrastive loss that bridges the gap between CLIP for paired cross-modal data and SupCon for uni-modal supervised contrastive learning.
  arXiv  Detail & Related papers  (2026-02-03T21:11:06Z)
• Disentangled Representation Learning via Modular Compositional Bias [19.244228209387163]
  We propose a compositional bias, a modular inductive bias decoupled from both objectives and architectures.<n>Our key insight is that different factors obey distinct recombination rules in the data distribution.<n>Our method shows competitive performance in both attribute and object disentanglement, and uniquely achieves joint disentanglement of global style and objects.
  arXiv  Detail & Related papers  (2025-10-24T12:46:19Z)
• Generalized Contrastive Learning for Universal Multimodal Retrieval [53.70202081784898]
  Cross-modal retrieval models (e.g., CLIP) show degraded performances with retrieving keys composed of fused image-text modality.<n>This paper proposes Generalized Contrastive Learning (GCL), a novel loss formulation that improves multimodal retrieval performance without the need for new dataset curation.
  arXiv  Detail & Related papers  (2025-09-30T01:25:04Z)
• IndiSeek learns information-guided disentangled representations [9.021908698915013]
  We introduce IndiSeek, a novel disentangled representation learning approach.<n>IndiSeek balances independence and completeness, enabling principled extraction of modality-specific features.<n>We demonstrate the effectiveness of IndiSeek on synthetic simulations, a CITE-seq dataset and multiple real-world multi-modal benchmarks.
  arXiv  Detail & Related papers  (2025-09-25T20:58:34Z)
• Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning [80.44084021062105]
  We propose a novel latent partial causal model for multimodal data, featuring two latent coupled variables, connected by an undirected edge, to represent the transfer of knowledge across modalities.<n>Under specific statistical assumptions, we establish an identifiability result, demonstrating that representations learned by multimodal contrastive learning correspond to the latent coupled variables up to a trivial transformation.<n>Experiments on a pre-trained CLIP model embodies disentangled representations, enabling few-shot learning and improving domain generalization across diverse real-world datasets.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• Learning Multimodal VAEs through Mutual Supervision [72.77685889312889]
  MEME combines information between modalities implicitly through mutual supervision. We demonstrate that MEME outperforms baselines on standard metrics across both partial and complete observation schemes.
  arXiv  Detail & Related papers  (2021-06-23T17:54:35Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.