Contrastive Factor Analysis

• URL: http://arxiv.org/abs/2407.21740v2
• Date: Thu, 1 Aug 2024 03:16:43 GMT
• Title: Contrastive Factor Analysis
• Authors: Zhibin Duan, Tiansheng Wen, Yifei Wang, Chen Zhu, Bo Chen, Mingyuan Zhou,
• Abstract summary: This paper introduces a novel Contrastive Factor Analysis framework. It aims to leverage factor analysis's advantageous properties within the realm of contrastive learning. To further leverage the interpretability properties of non-negative factor analysis, it is extended to a non-negative version.
• Score: 70.02770079785559
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Factor analysis, often regarded as a Bayesian variant of matrix factorization, offers superior capabilities in capturing uncertainty, modeling complex dependencies, and ensuring robustness. As the deep learning era arrives, factor analysis is receiving less and less attention due to their limited expressive ability. On the contrary, contrastive learning has emerged as a potent technique with demonstrated efficacy in unsupervised representational learning. While the two methods are different paradigms, recent theoretical analysis has revealed the mathematical equivalence between contrastive learning and matrix factorization, providing a potential possibility for factor analysis combined with contrastive learning. Motivated by the interconnectedness of contrastive learning, matrix factorization, and factor analysis, this paper introduces a novel Contrastive Factor Analysis framework, aiming to leverage factor analysis's advantageous properties within the realm of contrastive learning. To further leverage the interpretability properties of non-negative factor analysis, which can learn disentangled representations, contrastive factor analysis is extended to a non-negative version. Finally, extensive experimental validation showcases the efficacy of the proposed contrastive (non-negative) factor analysis methodology across multiple key properties, including expressiveness, robustness, interpretability, and accurate uncertainty estimation.

Related papers

• Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models [85.67870425656368]
  We introduce a unified causal model specifically designed for multimodal data. We show that multimodal contrastive representation learning excels at identifying latent coupled variables. Experiments demonstrate the robustness of our findings, even when the assumptions are violated.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Sensitivity Analysis of High-Dimensional Models with Correlated Inputs [0.0]
  The sensitivity of correlated parameters can not only differ in magnitude, but even the sign of the derivative-based index can be inverted. We demonstrate that the sensitivity of the correlated parameters can not only differ in magnitude, but even the sign of the derivative-based index can be inverted.
  arXiv  Detail & Related papers  (2023-05-31T14:48:54Z)
• Neuro-Causal Factor Analysis [18.176375611711396]
  We introduce a framework for Neuro-Causal Factor Analysis (NCFA) NCFA identifies factors via latent causal discovery methods and then uses a variational autoencoder (VAE) We evaluate NCFA on real and synthetic data sets, finding that it performs comparably to standard VAEs on data reconstruction tasks.
  arXiv  Detail & Related papers  (2023-05-31T12:41:20Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis [56.84237932819403]
  This paper aims to estimate and mitigate the bad effect of textual modality for strong OOD generalization. Inspired by this, we devise a model-agnostic counterfactual framework for multimodal sentiment analysis.
  arXiv  Detail & Related papers  (2022-07-24T03:57:40Z)
• On Finite-Sample Identifiability of Contrastive Learning-Based Nonlinear Independent Component Analysis [11.012445089716016]
  This work puts forth a finite-sample identifiability analysis of GCL-based nICA. Our framework judiciously combines the properties of the GCL loss function, statistical analysis, and numerical differentiation.
  arXiv  Detail & Related papers  (2022-06-14T04:59:08Z)
• Understanding Contrastive Learning Requires Incorporating Inductive Biases [64.56006519908213]
  Recent attempts to theoretically explain the success of contrastive learning on downstream tasks prove guarantees depending on properties of em augmentations and the value of em contrastive loss of representations. We demonstrate that such analyses ignore em inductive biases of the function class and training algorithm, even em provably leading to vacuous guarantees in some settings.
  arXiv  Detail & Related papers  (2022-02-28T18:59:20Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.