Revisiting Deep Generalized Canonical Correlation Analysis

• URL: http://arxiv.org/abs/2312.13455v1
• Date: Wed, 20 Dec 2023 22:15:10 GMT
• Title: Revisiting Deep Generalized Canonical Correlation Analysis
• Authors: Paris A. Karakasis, Nicholas D. Sidiropoulos
• Abstract summary: Canonical correlation analysis is a classic method for discovering latent co-variation that underpins two or more observed random vectors. Several extensions and variations of CCA have been proposed that have strengthened our capabilities in terms of revealing common random factors from multiview datasets. In this work, we first revisit the most recent deterministic extensions of deep CCA and highlight the strengths and limitations of these state-of-the-art methods.
• Score: 30.389620125859356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Canonical correlation analysis (CCA) is a classic statistical method for discovering latent co-variation that underpins two or more observed random vectors. Several extensions and variations of CCA have been proposed that have strengthened our capabilities in terms of revealing common random factors from multiview datasets. In this work, we first revisit the most recent deterministic extensions of deep CCA and highlight the strengths and limitations of these state-of-the-art methods. Some methods allow trivial solutions, while others can miss weak common factors. Others overload the problem by also seeking to reveal what is not common among the views -- i.e., the private components that are needed to fully reconstruct each view. The latter tends to overload the problem and its computational and sample complexities. Aiming to improve upon these limitations, we design a novel and efficient formulation that alleviates some of the current restrictions. The main idea is to model the private components as conditionally independent given the common ones, which enables the proposed compact formulation. In addition, we also provide a sufficient condition for identifying the common random factors. Judicious experiments with synthetic and real datasets showcase the validity of our claims and the effectiveness of the proposed approach.

Related papers

• Bayesian Joint Additive Factor Models for Multiview Learning [7.254731344123118]
  A motivating application arises in the context of precision medicine where multi-omics data are collected to correlate with clinical outcomes. We propose a joint additive factor regression model (JAFAR) with a structured additive design, accounting for shared and view-specific components. Prediction of time-to-labor onset from immunome, metabolome, and proteome data illustrates performance gains against state-of-the-art competitors.
  arXiv  Detail & Related papers  (2024-06-02T15:35:45Z)
• Causal Representation Learning Made Identifiable by Grouping of Observational Variables [8.157856010838382]
  Causal Representation Learning aims to learn a causal model for hidden features in a data-driven manner. Here, we show identifiability based on novel, weak constraints. We also propose a novel self-supervised estimation framework consistent with the model.
  arXiv  Detail & Related papers  (2023-10-24T10:38:02Z)
• Causality and Independence Enhancement for Biased Node Classification [56.38828085943763]
  We propose a novel Causality and Independence Enhancement (CIE) framework, applicable to various graph neural networks (GNNs) Our approach estimates causal and spurious features at the node representation level and mitigates the influence of spurious correlations. Our approach CIE not only significantly enhances the performance of GNNs but outperforms state-of-the-art debiased node classification methods.
  arXiv  Detail & Related papers  (2023-10-14T13:56:24Z)
• Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction [22.567754688492414]
  Cross-device Federated Learning (FL) faces significant challenges where low-end clients that could potentially make unique contributions are excluded from training large models due to their resource bottlenecks. Recent research efforts have focused on model-heterogeneous FL, by extracting reduced-size models from the global model and applying them to local clients accordingly. This paper presents a unifying framework for heterogeneous FL algorithms with online model extraction and provides a general convergence analysis for the first time.
  arXiv  Detail & Related papers  (2023-10-12T19:07:58Z)
• Endogenous Macrodynamics in Algorithmic Recourse [52.87956177581998]
  Existing work on Counterfactual Explanations (CE) and Algorithmic Recourse (AR) has largely focused on single individuals in a static environment. We show that many of the existing methodologies can be collectively described by a generalized framework. We then argue that the existing framework does not account for a hidden external cost of recourse, that only reveals itself when studying the endogenous dynamics of recourse at the group level.
  arXiv  Detail & Related papers  (2023-08-16T07:36:58Z)
• Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning [98.78136504619539]
  Causal Triplet is a causal representation learning benchmark featuring visually more complex scenes. We show that models built with the knowledge of disentangled or object-centric representations significantly outperform their distributed counterparts.
  arXiv  Detail & Related papers  (2023-01-12T17:43:38Z)
• The Causal Marginal Polytope for Bounding Treatment Effects [9.196779204457059]
  We propose a novel way to identify causal effects without constructing a global causal model. We enforce compatibility between marginals of a causal model and data, without constructing a global causal model. We call this collection of locally consistent marginals the causal marginal polytope.
  arXiv  Detail & Related papers  (2022-02-28T15:08:22Z)
• Effect Identification in Cluster Causal Diagrams [51.42809552422494]
  We introduce a new type of graphical model called cluster causal diagrams (for short, C-DAGs) C-DAGs allow for the partial specification of relationships among variables based on limited prior knowledge. We develop the foundations and machinery for valid causal inferences over C-DAGs.
  arXiv  Detail & Related papers  (2022-02-22T21:27:31Z)
• Provable Reinforcement Learning with a Short-Term Memory [68.00677878812908]
  We study a new subclass of POMDPs, whose latent states can be decoded by the most recent history of a short length $m$. In particular, in the rich-observation setting, we develop new algorithms using a novel "moment matching" approach with a sample complexity that scales exponentially. Our results show that a short-term memory suffices for reinforcement learning in these environments.
  arXiv  Detail & Related papers  (2022-02-08T16:39:57Z)
• Recursive Causal Structure Learning in the Presence of Latent Variables and Selection Bias [27.06618125828978]
  We consider the problem of learning the causal MAG of a system from observational data in the presence of latent variables and selection bias. We propose a novel computationally efficient constraint-based method that is sound and complete. We provide experimental results to compare the proposed approach with the state of the art on both synthetic and real-world structures.
  arXiv  Detail & Related papers  (2021-10-22T19:49:59Z)
• Causal Collaborative Filtering [50.22155187512759]
  Causal Collaborative Filtering is a framework for modeling causality in collaborative filtering and recommendation. We show that many traditional CF algorithms are actually special cases of CCF under simplified causal graphs. We propose a conditional intervention approach for $do$-operations so that we can estimate the user-item causal preference.
  arXiv  Detail & Related papers  (2021-02-03T04:16:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.