On Deep Instrumental Variables Estimate
- URL: http://arxiv.org/abs/2004.14954v1
- Date: Thu, 30 Apr 2020 17:03:00 GMT
- Title: On Deep Instrumental Variables Estimate
- Authors: Ruiqi Liu, Zuofeng Shang, Guang Cheng
- Abstract summary: "Deep Instrumental Variable (IV)" is a framework based on deep neural networks to address endogeneity.
We consider a two-stage estimator using deep neural networks in the linear instrumental variables model.
We show that the second-stage estimator achieves the semiparametric efficiency bound.
- Score: 21.003925102068298
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The endogeneity issue is fundamentally important as many empirical
applications may suffer from the omission of explanatory variables, measurement
error, or simultaneous causality. Recently, \cite{hllt17} propose a "Deep
Instrumental Variable (IV)" framework based on deep neural networks to address
endogeneity, demonstrating superior performances than existing approaches. The
aim of this paper is to theoretically understand the empirical success of the
Deep IV. Specifically, we consider a two-stage estimator using deep neural
networks in the linear instrumental variables model. By imposing a latent
structural assumption on the reduced form equation between endogenous variables
and instrumental variables, the first-stage estimator can automatically capture
this latent structure and converge to the optimal instruments at the minimax
optimal rate, which is free of the dimension of instrumental variables and thus
mitigates the curse of dimensionality. Additionally, in comparison with
classical methods, due to the faster convergence rate of the first-stage
estimator, the second-stage estimator has {a smaller (second order) estimation
error} and requires a weaker condition on the smoothness of the optimal
instruments. Given that the depth and width of the employed deep neural network
are well chosen, we further show that the second-stage estimator achieves the
semiparametric efficiency bound. Simulation studies on synthetic data and
application to automobile market data confirm our theory.
Related papers
- Sparse deep neural networks for nonparametric estimation in high-dimensional sparse regression [4.983567824636051]
This study combines nonparametric estimation and parametric sparse deep neural networks for the first time.
As nonparametric estimation of partial derivatives is of great significance for nonlinear variable selection, the current results show the promising future for the interpretability of deep neural networks.
arXiv Detail & Related papers (2024-06-26T07:41:41Z) - Semi-Supervised Deep Sobolev Regression: Estimation, Variable Selection
and Beyond [3.782392436834913]
We propose SDORE, a semi-supervised deep Sobolev regressor, for the nonparametric estimation of the underlying regression function and its gradient.
We conduct a comprehensive analysis of the convergence rates of SDORE and establish a minimax optimal rate for the regression function.
We also derive a convergence rate for the associated plug-in gradient estimator, even in the presence of significant domain shift.
arXiv Detail & Related papers (2024-01-09T13:10:30Z) - Structured Radial Basis Function Network: Modelling Diversity for
Multiple Hypotheses Prediction [51.82628081279621]
Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions.
A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems.
It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
arXiv Detail & Related papers (2023-09-02T01:27:53Z) - Structured Optimal Variational Inference for Dynamic Latent Space Models [16.531262817315696]
We consider a latent space model for dynamic networks, where our objective is to estimate the pairwise inner products plus the intercept of the latent positions.
To balance posterior inference and computational scalability, we consider a structured mean-field variational inference framework.
arXiv Detail & Related papers (2022-09-29T22:10:42Z) - Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
Implicit neural networks show improved accuracy and significant reduction in memory consumption.
They can suffer from ill-posedness and convergence instability.
This paper provides a new framework to design well-posed and robust implicit neural networks.
arXiv Detail & Related papers (2021-06-06T18:05:02Z) - Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
Inferring the parameters of a model based on experimental observations is central to the scientific method.
A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations.
We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
arXiv Detail & Related papers (2021-02-12T12:23:13Z) - Multi-fidelity Bayesian Neural Networks: Algorithms and Applications [0.0]
We propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity.
We apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs)
arXiv Detail & Related papers (2020-12-19T02:03:53Z) - High Dimensional Level Set Estimation with Bayesian Neural Network [58.684954492439424]
This paper proposes novel methods to solve the high dimensional Level Set Estimation problems using Bayesian Neural Networks.
For each problem, we derive the corresponding theoretic information based acquisition function to sample the data points.
Numerical experiments on both synthetic and real-world datasets show that our proposed method can achieve better results compared to existing state-of-the-art approaches.
arXiv Detail & Related papers (2020-12-17T23:21:53Z) - SODEN: A Scalable Continuous-Time Survival Model through Ordinary
Differential Equation Networks [14.564168076456822]
We propose a flexible model for survival analysis using neural networks along with scalable optimization algorithms.
We demonstrate the effectiveness of the proposed method in comparison to existing state-of-the-art deep learning survival analysis models.
arXiv Detail & Related papers (2020-08-19T19:11:25Z) - Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks [78.76880041670904]
In neural networks with binary activations and or binary weights the training by gradient descent is complicated.
We propose a new method for this estimation problem combining sampling and analytic approximation steps.
We experimentally show higher accuracy in gradient estimation and demonstrate a more stable and better performing training in deep convolutional models.
arXiv Detail & Related papers (2020-06-04T21:51:21Z) - Communication-Efficient Distributed Stochastic AUC Maximization with
Deep Neural Networks [50.42141893913188]
We study a distributed variable for large-scale AUC for a neural network as with a deep neural network.
Our model requires a much less number of communication rounds and still a number of communication rounds in theory.
Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
arXiv Detail & Related papers (2020-05-05T18:08:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.