The Variational Method of Moments
- URL: http://arxiv.org/abs/2012.09422v4
- Date: Wed, 22 Mar 2023 21:49:23 GMT
- Title: The Variational Method of Moments
- Authors: Andrew Bennett, Nathan Kallus
- Abstract summary: conditional moment problem is a powerful formulation for describing structural causal parameters in terms of observables.
Motivated by a variational minimax reformulation of OWGMM, we define a very general class of estimators for the conditional moment problem.
We provide algorithms for valid statistical inference based on the same kind of variational reformulations.
- Score: 65.91730154730905
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The conditional moment problem is a powerful formulation for describing
structural causal parameters in terms of observables, a prominent example being
instrumental variable regression. A standard approach reduces the problem to a
finite set of marginal moment conditions and applies the optimally weighted
generalized method of moments (OWGMM), but this requires we know a finite set
of identifying moments, can still be inefficient even if identifying, or can be
theoretically efficient but practically unwieldy if we use a growing sieve of
moment conditions. Motivated by a variational minimax reformulation of OWGMM,
we define a very general class of estimators for the conditional moment
problem, which we term the variational method of moments (VMM) and which
naturally enables controlling infinitely-many moments. We provide a detailed
theoretical analysis of multiple VMM estimators, including ones based on kernel
methods and neural nets, and provide conditions under which these are
consistent, asymptotically normal, and semiparametrically efficient in the full
conditional moment model. We additionally provide algorithms for valid
statistical inference based on the same kind of variational reformulations,
both for kernel- and neural-net-based varieties. Finally, we demonstrate the
strong performance of our proposed estimation and inference algorithms in a
detailed series of synthetic experiments.
Related papers
- Online Variational Sequential Monte Carlo [49.97673761305336]
We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference.
Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
arXiv Detail & Related papers (2023-12-19T21:45:38Z) - Monotonic Neural Ordinary Differential Equation: Time-series Forecasting
for Cumulative Data [9.03818193356305]
We propose a principled approach called Monotonic neural Ordinary Differential Equation (MODE) within the framework of neural ordinary differential equations.
By leveraging MODE, we are able to effectively capture and represent the monotonicity and irregularity in practical cumulative data.
We demonstrate that MODE outperforms state-of-the-art methods, showcasing its ability to handle both monotonicity and irregularity in cumulative data.
arXiv Detail & Related papers (2023-09-23T18:40:10Z) - Spectral Representation Learning for Conditional Moment Models [33.34244475589745]
We propose a procedure that automatically learns representations with controlled measures of ill-posedness.
Our method approximates a linear representation defined by the spectral decomposition of a conditional expectation operator.
We show this representation can be efficiently estimated from data, and establish L2 consistency for the resulting estimator.
arXiv Detail & Related papers (2022-10-29T07:48:29Z) - Functional Generalized Empirical Likelihood Estimation for Conditional
Moment Restrictions [19.39005034948997]
We propose a new estimation method based on generalized empirical likelihood (GEL)
GEL provides a more general framework and has been shown to enjoy favorable small-sample properties compared to GMM-based estimators.
We provide kernel- and neural network-based implementations of the estimator, which achieve state-of-the-art empirical performance on two conditional moment restriction problems.
arXiv Detail & Related papers (2022-07-11T11:02:52Z) - Continuous-Time Modeling of Counterfactual Outcomes Using Neural
Controlled Differential Equations [84.42837346400151]
Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare.
Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions.
We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
arXiv Detail & Related papers (2022-06-16T17:15:15Z) - Accelerated and instance-optimal policy evaluation with linear function
approximation [17.995515643150657]
Existing algorithms fail to match at least one of these lower bounds.
We develop an accelerated, variance-reduced fast temporal difference algorithm that simultaneously matches both lower bounds and attains a strong notion of instance-optimality.
arXiv Detail & Related papers (2021-12-24T17:21:04Z) - Amortized Conditional Normalized Maximum Likelihood: Reliable Out of
Distribution Uncertainty Estimation [99.92568326314667]
We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation.
Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle.
We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
arXiv Detail & Related papers (2020-11-05T08:04:34Z) - Density Fixing: Simple yet Effective Regularization Method based on the
Class Prior [2.3859169601259347]
We propose a framework of regularization methods, called density-fixing, that can be used commonly for supervised and semi-supervised learning.
Our proposed regularization method improves the generalization performance by forcing the model to approximate the class's prior distribution or the frequency of occurrence.
arXiv Detail & Related papers (2020-07-08T04:58:22Z) - Is Temporal Difference Learning Optimal? An Instance-Dependent Analysis [102.29671176698373]
We address the problem of policy evaluation in discounted decision processes, and provide Markov-dependent guarantees on the $ell_infty$error under a generative model.
We establish both and non-asymptotic versions of local minimax lower bounds for policy evaluation, thereby providing an instance-dependent baseline by which to compare algorithms.
arXiv Detail & Related papers (2020-03-16T17:15:28Z) - Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic
Perspectives [97.16266088683061]
The article rigorously establishes why symplectic discretization schemes are important for momentum-based optimization algorithms.
It provides a characterization of algorithms that exhibit accelerated convergence.
arXiv Detail & Related papers (2020-02-28T00:32:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.