Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness

• URL: http://arxiv.org/abs/2106.08161v1
• Date: Tue, 15 Jun 2021 14:04:55 GMT
• Title: Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness
• Authors: Adam Foster, \'Arpi Vez\'er, Craig A Glastonbury, P\'aid\'i Creed, Sam Abujudeh, Aaron Sim
• Abstract summary: Learn meaningful representations of data that can address challenges such as batch effect correction, data integration and counterfactual inference. Adopting a Conditional VAE framework, we identify the mathematical principle that unites these challenges: learning a representation that is marginally independent of a condition variable. We propose the Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment penalty to enforce this independence.
• Score: 0.8528401618469597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning meaningful representations of data that can address challenges such as batch effect correction, data integration and counterfactual inference is a central problem in many domains including computational biology. Adopting a Conditional VAE framework, we identify the mathematical principle that unites these challenges: learning a representation that is marginally independent of a condition variable. We therefore propose the Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment penalty to enforce this independence. This penalty is defined in terms of mixtures of the variational posteriors themselves, unlike prior work which uses external discrepancy measures such as MMD to ensure independence in latent space. We show that CoMP has attractive theoretical properties compared to previous approaches, especially when there is complex global structure in latent space. We further demonstrate state of the art performance on a number of real-world problems, including the challenging tasks of aligning human tumour samples with cancer cell-lines and performing counterfactual inference on single-cell RNA sequencing data. Incidentally, we find parallels with the fair representation learning literature, and demonstrate CoMP has competitive performance in learning fair yet expressive latent representations.

Related papers

• Nonparametric Identification and Inference for Counterfactual Distributions with Confounding [6.997978440999076]
  We propose nonparametric identification and semiparametric estimation of joint potential outcome in the presence of confounding.<n>By bridging classical semiparametric theory with modern representation learning, this work provides a robust statistical foundation for distributional and counterfactual inference in complex causal systems.
  arXiv  Detail & Related papers  (2026-02-17T05:00:13Z)
• Towards Inference-time Scaling for Continuous Space Reasoning [55.40260529506702]
  Inference-time scaling has proven effective for text-based reasoning in large language models.<n>This paper investigates whether such established techniques can be successfully adapted to reasoning in the continuous space.<n>We demonstrate the feasibility of generating diverse reasoning paths through dropout-based sampling.
  arXiv  Detail & Related papers  (2025-10-14T05:53:41Z)
• CM$^3$: Calibrating Multimodal Recommendation [10.09576389984858]
  This study revisits the alignment and uniformity properties within the context of multimodal recommender systems.<n>We propose a more nuanced approach wherein items with similar multimodal attributes converge toward proximal representations within the hyperspheric manifold.<n>We also introduce a Spherical B'ezier method designed to integrate an arbitrary number of modalities while ensuring that the resulting fused features are constrained to the same hyperspherical manifold.
  arXiv  Detail & Related papers  (2025-08-02T06:44:59Z)
• A Sample Efficient Conditional Independence Test in the Presence of Discretization [54.047334792855345]
  Conditional Independence (CI) tests directly to discretized data can lead to incorrect conclusions.<n>Recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data.<n>Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process.
  arXiv  Detail & Related papers  (2025-06-10T12:41:26Z)
• Unsupervised Structural-Counterfactual Generation under Domain Shift [0.0]
  We present a novel generative modeling challenge: generating counterfactual samples in a target domain based on factual observations from a source domain. Our framework combines the posterior distribution of effect-intrinsic variables from the source domain with the prior distribution of domain-intrinsic variables from the target domain to synthesize the desired counterfactuals.
  arXiv  Detail & Related papers  (2025-02-17T16:48:16Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective. The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning. The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• Self-Distilled Disentangled Learning for Counterfactual Prediction [49.84163147971955]
  We propose the Self-Distilled Disentanglement framework, known as $SD2$. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs. Our experiments, conducted on both synthetic and real-world datasets, confirm the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-06-09T16:58:19Z)
• Counterfactual Fairness through Transforming Data Orthogonal to Bias [7.109458605736819]
  We propose a novel data pre-processing algorithm, Orthogonal to Bias (OB) OB is designed to eliminate the influence of a group of continuous sensitive variables, thus promoting counterfactual fairness in machine learning applications. OB is model-agnostic, making it applicable to a wide range of machine learning models and tasks.
  arXiv  Detail & Related papers  (2024-03-26T16:40:08Z)
• Mapping the Multiverse of Latent Representations [17.2089620240192]
  PRESTO is a principled framework for mapping the multiverse of machine-learning models that rely on latent representations. Our framework uses persistent homology to characterize the latent spaces arising from different combinations of diverse machine-learning methods.
  arXiv  Detail & Related papers  (2024-02-02T15:54:53Z)
• Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approach [10.27974860479791]
  In many applications where the data exhibit temporal dependencies, the corresponding empirical processes are much less understood. We present a general bound on the expected supremum of empirical processes under standard $beta/rho$-mixing assumptions. We show that even under long-range dependence, it is possible to attain the same rates as in the i.i.d. setting.
  arXiv  Detail & Related papers  (2024-01-17T05:08:37Z)
• Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning [60.058083574671834]
  This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation. For heterogeneous issue, we leverage irrelevant unlabeled public data for communication. For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation.
  arXiv  Detail & Related papers  (2023-09-28T09:32:27Z)
• Conditional Kernel Imitation Learning for Continuous State Environments [9.750698192309978]
  We introduce a novel conditional kernel density estimation-based imitation learning framework. We show consistently superior empirical performance over many state-of-the-art IL algorithms.
  arXiv  Detail & Related papers  (2023-08-24T05:26:42Z)
• Towards Characterizing Domain Counterfactuals For Invertible Latent Causal Models [15.817239008727789]
  In this work, we analyze a specific type of causal query called domain counterfactuals, which hypothesizes what a sample would have looked like if it had been generated in a different domain. We show that recovering the latent Structural Causal Model (SCM) is unnecessary for estimating domain counterfactuals. We also develop a theoretically grounded practical algorithm that simplifies the modeling process to generative model estimation.
  arXiv  Detail & Related papers  (2023-06-20T04:19:06Z)
• Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer [72.5190560787569]
  In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets. Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions. This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes.
  arXiv  Detail & Related papers  (2020-11-25T00:13:11Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
  We show that effective estimation can still be achieved in important applications. Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions. The resulting algorithm, GenDICE, is straightforward and effective.
  arXiv  Detail & Related papers  (2020-02-21T00:27:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.