Spectral Representation for Causal Estimation with Hidden Confounders

• URL: http://arxiv.org/abs/2407.10448v1
• Date: Mon, 15 Jul 2024 05:39:56 GMT
• Title: Spectral Representation for Causal Estimation with Hidden Confounders
• Authors: Tongzheng Ren, Haotian Sun, Antoine Moulin, Arthur Gretton, Bo Dai,
• Abstract summary: We address the problem of causal effect estimation where hidden confounders are present. Our approach uses a singular value decomposition of a conditional expectation operator, followed by a saddle-point optimization problem.
• Score: 33.148766692274215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the problem of causal effect estimation where hidden confounders are present, with a focus on two settings: instrumental variable regression with additional observed confounders, and proxy causal learning. Our approach uses a singular value decomposition of a conditional expectation operator, followed by a saddle-point optimization problem, which, in the context of IV regression, can be thought of as a neural net generalization of the seminal approach due to Darolles et al. [2011]. Saddle-point formulations have gathered considerable attention recently, as they can avoid double sampling bias and are amenable to modern function approximation methods. We provide experimental validation in various settings, and show that our approach outperforms existing methods on common benchmarks.

Related papers

• On Sampling Strategies for Spectral Model Sharding [7.185534285278903]
  In this work, we present two sampling strategies for such sharding. The first produces unbiased estimators of the original weights, while the second aims to minimize the squared approximation error. We demonstrate that both of these methods can lead to improved performance on various commonly used datasets.
  arXiv  Detail & Related papers  (2024-10-31T16:37:25Z)
• Long-Sequence Recommendation Models Need Decoupled Embeddings [49.410906935283585]
  We identify and characterize a neglected deficiency in existing long-sequence recommendation models. A single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes. We propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are learned separately to fully decouple attention and representation.
  arXiv  Detail & Related papers  (2024-10-03T15:45:15Z)
• Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients [0.3277163122167434]
  We show how to formulate a functional gradient descent algorithm to tackle NPIV regression by directly minimizing the populational risk. We provide theoretical support in the form of bounds on the excess risk, and conduct numerical experiments showcasing our method's superior stability and competitive performance. This algorithm enables flexible estimator choices, such as neural networks or kernel based methods, as well as non-quadratic loss functions.
  arXiv  Detail & Related papers  (2024-02-08T12:50:38Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Distributionally Robust Causal Inference with Observational Data [4.8986598953553555]
  We consider the estimation of average treatment effects in observational studies without the standard assumption of unconfoundedness. We propose a new framework of robust causal inference under the general observational study setting with the possible existence of unobserved confounders.
  arXiv  Detail & Related papers  (2022-10-15T16:02:33Z)
• Double logistic regression approach to biased positive-unlabeled data [3.6594988197536344]
  We consider parametric approach to the problem of joint estimation of posterior probability and propensity score functions. Motivated by this, we propose two approaches to their estimation: joint maximum likelihood method and the second approach based on alternating alternating expressions. Our experimental results show that the proposed methods are comparable or better than the existing methods based on Expectation-Maximisation scheme.
  arXiv  Detail & Related papers  (2022-09-16T08:32:53Z)
• Generalization bounds and algorithms for estimating conditional average treatment effect of dosage [13.867315751451494]
  We investigate the task of estimating the conditional average causal effect of treatment-dosage pairs from a combination of observational data and assumptions on the causal relationships in the underlying system. This has been a longstanding challenge for fields of study such as epidemiology or economics that require a treatment-dosage pair to make decisions. We show empirically new state-of-the-art performance results across several benchmark datasets for this problem.
  arXiv  Detail & Related papers  (2022-05-29T15:26:59Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
  Learning from implicit feedback is challenging because of the difficult nature of the one-class problem. Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem. We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
  arXiv  Detail & Related papers  (2021-05-11T03:38:16Z)
• Supervised PCA: A Multiobjective Approach [70.99924195791532]
  Methods for supervised principal component analysis (SPCA) We propose a new method for SPCA that addresses both of these objectives jointly. Our approach accommodates arbitrary supervised learning losses and, through a statistical reformulation, provides a novel low-rank extension of generalized linear models.
  arXiv  Detail & Related papers  (2020-11-10T18:46:58Z)
• GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
  We show that effective estimation can still be achieved in important applications. Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions. The resulting algorithm, GenDICE, is straightforward and effective.
  arXiv  Detail & Related papers  (2020-02-21T00:27:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.