Regularizing towards Causal Invariance: Linear Models with Proxies
- URL: http://arxiv.org/abs/2103.02477v1
- Date: Wed, 3 Mar 2021 15:39:35 GMT
- Title: Regularizing towards Causal Invariance: Linear Models with Proxies
- Authors: Michael Oberst, Nikolaj Thams, Jonas Peters, David Sontag
- Abstract summary: We show that a single proxy can be used to create estimators that are prediction optimal under interventions of bounded strength.
We show how to extend these estimators to scenarios where additional information about the "test time" intervention is available during training.
- Score: 7.953401800573514
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose a method for learning linear models whose predictive performance
is robust to causal interventions on unobserved variables, when noisy proxies
of those variables are available. Our approach takes the form of a
regularization term that trades off between in-distribution performance and
robustness to interventions. Under the assumption of a linear structural causal
model, we show that a single proxy can be used to create estimators that are
prediction optimal under interventions of bounded strength. This strength
depends on the magnitude of the measurement noise in the proxy, which is, in
general, not identifiable. In the case of two proxy variables, we propose a
modified estimator that is prediction optimal under interventions up to a known
strength. We further show how to extend these estimators to scenarios where
additional information about the "test time" intervention is available during
training. We evaluate our theoretical findings in synthetic experiments and
using real data of hourly pollution levels across several cities in China.
Related papers
- Quantification of Predictive Uncertainty via Inference-Time Sampling [57.749601811982096]
We propose a post-hoc sampling strategy for estimating predictive uncertainty accounting for data ambiguity.
The method can generate different plausible outputs for a given input and does not assume parametric forms of predictive distributions.
arXiv Detail & Related papers (2023-08-03T12:43:21Z) - Causality-oriented robustness: exploiting general additive interventions [3.871660145364189]
In this paper, we focus on causality-oriented robustness and propose Distributional Robustness via Invariant Gradients (DRIG)
In a linear setting, we prove that DRIG yields predictions that are robust among a data-dependent class of distribution shifts.
We extend our approach to the semi-supervised domain adaptation setting to further improve prediction performance.
arXiv Detail & Related papers (2023-07-18T16:22:50Z) - Bayesian Sparse Regression for Mixed Multi-Responses with Application to
Runtime Metrics Prediction in Fog Manufacturing [6.288767115532775]
Fog manufacturing can greatly enhance traditional manufacturing systems through distributed computation Fog units.
It is known that the predictive offloading methods highly depend on accurate prediction and uncertainty quantification of runtime performance metrics.
We propose a Bayesian sparse regression for multivariate mixed responses to enhance the prediction of runtime performance metrics.
arXiv Detail & Related papers (2022-10-10T16:14:08Z) - Time Series Prediction under Distribution Shift using Differentiable
Forgetting [0.0]
We frame time series prediction under distribution shift as a weighted empirical risk minimisation problem.
The weighting of previous observations in the empirical risk is determined by a forgetting mechanism.
We propose a gradient-based learning method for the parameters of the forgetting mechanism.
arXiv Detail & Related papers (2022-07-23T10:32:37Z) - TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance.
We propose a versatile method that estimates joint distributions using an attention-based decoder.
We show that our model produces state-of-the-art predictions on several real-world datasets.
arXiv Detail & Related papers (2022-02-07T21:37:29Z) - Causality and Generalizability: Identifiability and Learning Methods [0.0]
This thesis contributes to the research areas concerning the estimation of causal effects, causal structure learning, and distributionally robust prediction methods.
We present novel and consistent linear and non-linear causal effects estimators in instrumental variable settings that employ data-dependent mean squared prediction error regularization.
We propose a general framework for distributional robustness with respect to intervention-induced distributions.
arXiv Detail & Related papers (2021-10-04T13:12:11Z) - Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
We describe two types of forecasting problems: regular grid-based and graph-based.
We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory.
Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
arXiv Detail & Related papers (2021-05-25T14:35:46Z) - Learning Interpretable Deep State Space Model for Probabilistic Time
Series Forecasting [98.57851612518758]
Probabilistic time series forecasting involves estimating the distribution of future based on its history.
We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks.
We show in experiments that our model produces accurate and sharp probabilistic forecasts.
arXiv Detail & Related papers (2021-01-31T06:49:33Z) - Counterfactual Predictions under Runtime Confounding [74.90756694584839]
We study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data.
We propose a doubly-robust procedure for learning counterfactual prediction models in this setting.
arXiv Detail & Related papers (2020-06-30T15:49:05Z) - Machine learning for causal inference: on the use of cross-fit
estimators [77.34726150561087]
Doubly-robust cross-fit estimators have been proposed to yield better statistical properties.
We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE)
When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
arXiv Detail & Related papers (2020-04-21T23:09:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.