Propensity score models are better when post-calibrated
- URL: http://arxiv.org/abs/2211.01221v1
- Date: Wed, 2 Nov 2022 16:01:03 GMT
- Title: Propensity score models are better when post-calibrated
- Authors: Rom Gutman, Ehud Karavani, Yishai Shimoni
- Abstract summary: Post-calibration reduces the error in effect estimation for expressive uncalibrated statistical estimators.
Given the improvement in effect estimation and that post-calibration is computationally cheap, we recommend it will be adopted when modelling propensity scores with expressive models.
- Score: 0.32228025627337864
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Theoretical guarantees for causal inference using propensity scores are
partly based on the scores behaving like conditional probabilities. However,
scores between zero and one, especially when outputted by flexible statistical
estimators, do not necessarily behave like probabilities. We perform a
simulation study to assess the error in estimating the average treatment effect
before and after applying a simple and well-established post-processing method
to calibrate the propensity scores. We find that post-calibration reduces the
error in effect estimation for expressive uncalibrated statistical estimators,
and that this improvement is not mediated by better balancing. The larger the
initial lack of calibration, the larger the improvement in effect estimation,
with the effect on already-calibrated estimators being very small. Given the
improvement in effect estimation and that post-calibration is computationally
cheap, we recommend it will be adopted when modelling propensity scores with
expressive models.
Related papers
- C-Learner: Constrained Learning for Causal Inference and Semiparametric Statistics [5.395560682099634]
We propose a novel debiased estimator that achieves stable plug-in estimates with desirable properties.
Our constrained learning framework solves for the best plug-in estimator under the constraint that the first-order error with respect to the plugged-in quantity is zero.
Our estimator outperforms one-step estimation and targeting in challenging settings with limited overlap between treatment and control, and performs comparably otherwise.
arXiv Detail & Related papers (2024-05-15T16:38:28Z) - Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data [13.069717985067937]
We propose a weighted rolling-validation procedure, an online variant of leave-one-out cross-validation.
Similar to batch cross-validation, it can boost base estimators to achieve a better, adaptive convergence rate.
arXiv Detail & Related papers (2023-10-18T17:52:57Z) - Calibrated and Conformal Propensity Scores for Causal Effect Estimation [10.209143402485406]
We argue that the probabilistic output of a learned propensity score model should be calibrated.
Calibrated propensity scores improve the speed of GWAS analysis by more than two-fold.
arXiv Detail & Related papers (2023-06-01T06:26:26Z) - Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores.
We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
arXiv Detail & Related papers (2023-02-23T18:57:14Z) - Learning to Estimate Without Bias [57.82628598276623]
Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models.
In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints.
A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
arXiv Detail & Related papers (2021-10-24T10:23:51Z) - Expected Validation Performance and Estimation of a Random Variable's
Maximum [48.83713377993604]
We analyze three statistical estimators for expected validation performance.
We find the unbiased estimator has the highest variance, and the estimator with the smallest variance has the largest bias.
We find that the two biased estimators lead to the fewest incorrect conclusions.
arXiv Detail & Related papers (2021-10-01T18:48:47Z) - Near-optimal inference in adaptive linear regression [60.08422051718195]
Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner.
We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation.
We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
arXiv Detail & Related papers (2021-07-05T21:05:11Z) - Increasing the efficiency of randomized trial estimates via linear
adjustment for a prognostic score [59.75318183140857]
Estimating causal effects from randomized experiments is central to clinical research.
Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
arXiv Detail & Related papers (2020-12-17T21:10:10Z) - Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient
Estimator [93.05919133288161]
We show that the variance of the straight-through variant of the popular Gumbel-Softmax estimator can be reduced through Rao-Blackwellization.
This provably reduces the mean squared error.
We empirically demonstrate that this leads to variance reduction, faster convergence, and generally improved performance in two unsupervised latent variable models.
arXiv Detail & Related papers (2020-10-09T22:54:38Z) - Semi-supervised learning and the question of true versus estimated
propensity scores [0.456877715768796]
We propose a simple procedure that reconciles the strong intuition that a known propensity functions should be useful for estimating treatment effects.
Further, simulation studies suggest that direct regression may be preferable to inverse-propensity weight estimators in many circumstances.
arXiv Detail & Related papers (2020-09-14T04:13:12Z) - Distributional robustness of K-class estimators and the PULSE [4.56877715768796]
We prove that the classical K-class estimator satisfies such optimality by establishing a connection between K-class estimators and anchor regression.
We show that it can be computed efficiently as a data-driven simulation K-class estimator.
There are several settings including weak instrument settings, where it outperforms other estimators.
arXiv Detail & Related papers (2020-05-07T09:39:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.