Multi-View Causal Representation Learning with Partial Observability

• URL: http://arxiv.org/abs/2311.04056v2
• Date: Fri, 8 Mar 2024 15:43:50 GMT
• Title: Multi-View Causal Representation Learning with Partial Observability
• Authors: Dingling Yao, Danru Xu, S\'ebastien Lachapelle, Sara Magliacane, Perouz Taslakian, Georg Martius, Julius von K\"ugelgen and Francesco Locatello
• Abstract summary: We present a unified framework for studying identifiability of representations learned from simultaneously observed views. We prove that the information shared across all subsets of any number of views can be learned up to a smooth bijection using contrastive learning. We experimentally validate our claims on numerical, image, and multi-modal data sets.
• Score: 36.37049791756438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a unified framework for studying the identifiability of representations learned from simultaneously observed views, such as different data modalities. We allow a partially observed setting in which each view constitutes a nonlinear mixture of a subset of underlying latent variables, which can be causally related. We prove that the information shared across all subsets of any number of views can be learned up to a smooth bijection using contrastive learning and a single encoder per view. We also provide graphical criteria indicating which latent variables can be identified through a simple set of rules, which we refer to as identifiability algebra. Our general framework and theoretical results unify and extend several previous works on multi-view nonlinear ICA, disentanglement, and causal representation learning. We experimentally validate our claims on numerical, image, and multi-modal data sets. Further, we demonstrate that the performance of prior methods is recovered in different special cases of our setup. Overall, we find that access to multiple partial views enables us to identify a more fine-grained representation, under the generally milder assumption of partial observability.

Related papers

• Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models [85.67870425656368]
  We introduce a unified causal model specifically designed for multimodal data. We show that multimodal contrastive representation learning excels at identifying latent coupled variables. Experiments demonstrate the robustness of our findings, even when the assumptions are violated.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Multi-view Fuzzy Representation Learning with Rules based Model [25.997490574254172]
  Unsupervised multi-view representation learning has been extensively studied for mining multi-view data. This paper proposes a new multi-view fuzzy representation learning method based on the interpretable Takagi-Sugeno-Kang fuzzy system (MVRL_FS)
  arXiv  Detail & Related papers  (2023-09-20T17:13:15Z)
• TCGF: A unified tensorized consensus graph framework for multi-view representation learning [27.23929515170454]
  This paper proposes a universal multi-view representation learning framework named Consensus Graph Framework (TCGF) It first provides a unified framework for existing multi-view works to exploit the representations for individual view. Then, stacks them into a tensor under alignment basics as a high-order representation, allowing for the smooth propagation of consistency.
  arXiv  Detail & Related papers  (2023-09-14T19:29:14Z)
• Disentangling Multi-view Representations Beyond Inductive Bias [32.15900989696017]
  We propose a novel multi-view representation disentangling method that ensures both interpretability and generalizability of the resulting representations. Our experiments on four multi-view datasets demonstrate that our proposed method outperforms 12 comparison methods in terms of clustering and classification performance.
  arXiv  Detail & Related papers  (2023-08-03T09:09:28Z)
• Identifiability Results for Multimodal Contrastive Learning [72.15237484019174]
  We show that it is possible to recover shared factors in a more general setup than the multi-view setting studied previously. Our work provides a theoretical basis for multimodal representation learning and explains in which settings multimodal contrastive learning can be effective in practice.
  arXiv  Detail & Related papers  (2023-03-16T09:14:26Z)
• Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data [52.491074276133325]
  Multi-view representation learning has developed rapidly over the past decades and has been applied in many fields. We propose a new cross-view graph contrastive learning framework, which integrates multi-view information to align data and learn latent representations. Experiments conducted on several real datasets demonstrate the effectiveness of the proposed method on the clustering and classification tasks.
  arXiv  Detail & Related papers  (2022-11-08T09:19:32Z)
• Latent Heterogeneous Graph Network for Incomplete Multi-View Learning [57.49776938934186]
  We propose a novel Latent Heterogeneous Graph Network (LHGN) for incomplete multi-view learning. By learning a unified latent representation, a trade-off between consistency and complementarity among different views is implicitly realized. To avoid any inconsistencies between training and test phase, a transductive learning technique is applied based on graph learning for classification tasks.
  arXiv  Detail & Related papers  (2022-08-29T15:14:21Z)
• Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering [20.412896884905496]
  We present a novel VAE-based multi-view clustering framework (Multi-VAE) We define a view-common variable and multiple view-peculiar variables in the generative model. By controlling the mutual information capacity to disentangle the view-common and view-peculiar representations, continuous visual information of multiple views can be separated.
  arXiv  Detail & Related papers  (2021-06-21T16:23:28Z)
• Latent Correlation-Based Multiview Learning and Self-Supervision: A Unifying Perspective [41.80156041871873]
  This work puts forth a theory-backed framework for unsupervised multiview learning. Our development starts with proposing a multiview model, where each view is a nonlinear mixture of shared and private components. In addition, the private information in each view can be provably disentangled from the shared using proper regularization design.
  arXiv  Detail & Related papers  (2021-06-14T00:12:36Z)
• Deep Partial Multi-View Learning [94.39367390062831]
  We propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets) We fifirst provide a formal defifinition of completeness and versatility for multi-view representation. We then theoretically prove the versatility of the learned latent representations.
  arXiv  Detail & Related papers  (2020-11-12T02:29:29Z)
• Weakly-Supervised Disentanglement Without Compromises [53.55580957483103]
  Intelligent agents should be able to learn useful representations by observing changes in their environment. We model such observations as pairs of non-i.i.d. images sharing at least one of the underlying factors of variation. We show that only knowing how many factors have changed, but not which ones, is sufficient to learn disentangled representations.
  arXiv  Detail & Related papers  (2020-02-07T16:39:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.