Coordinated Double Machine Learning

• URL: http://arxiv.org/abs/2206.00885v1
• Date: Thu, 2 Jun 2022 05:56:21 GMT
• Title: Coordinated Double Machine Learning
• Authors: Nitai Fingerhut, Matteo Sesia, Yaniv Romano
• Abstract summary: This paper argues that a carefully coordinated learning algorithm for deep neural networks may reduce the estimation bias. The improved empirical performance of the proposed method is demonstrated through numerical experiments on both simulated and real data.
• Score: 8.808993671472349
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Double machine learning is a statistical method for leveraging complex black-box models to construct approximately unbiased treatment effect estimates given observational data with high-dimensional covariates, under the assumption of a partially linear model. The idea is to first fit on a subset of the samples two non-linear predictive models, one for the continuous outcome of interest and one for the observed treatment, and then to estimate a linear coefficient for the treatment using the remaining samples through a simple orthogonalized regression. While this methodology is flexible and can accommodate arbitrary predictive models, typically trained independently of one another, this paper argues that a carefully coordinated learning algorithm for deep neural networks may reduce the estimation bias. The improved empirical performance of the proposed method is demonstrated through numerical experiments on both simulated and real data.

Related papers

• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Addressing Data Scarcity in Optical Matrix Multiplier Modeling Using Transfer Learning [0.0]
  We present and experimentally evaluate using transfer learning to address experimental data scarcity. Our approach involves pre-training the model using synthetic data generated from a less accurate analytical model. We achieve 1 dB root-mean-square error on the matrix weights implemented by a 3x3 photonic chip while using only 25% of the available data.
  arXiv  Detail & Related papers  (2023-08-10T07:33:00Z)
• Exploiting Observation Bias to Improve Matrix Completion [16.57405742112833]
  We consider a variant of matrix completion where entries are revealed in a biased manner. The goal is to exploit the shared information between the bias and the outcome of interest to improve predictions. We find that with this two-stage algorithm, the estimates have 30x smaller mean squared error compared to traditional matrix completion methods.
  arXiv  Detail & Related papers  (2023-06-07T20:48:35Z)
• Compositional Score Modeling for Simulation-based Inference [28.422049267537965]
  We introduce a new method based on conditional score modeling that enjoys the benefits of both approaches. Our approach is sample-efficient, can naturally aggregate multiple observations at inference time, and avoids the drawbacks of standard inference methods.
  arXiv  Detail & Related papers  (2022-09-28T17:08:31Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Joint Dimensionality Reduction for Separable Embedding Estimation [43.22422640265388]
  Low-dimensional embeddings for data from disparate sources play critical roles in machine learning, multimedia information retrieval, and bioinformatics. We propose a supervised dimensionality reduction method that learns linear embeddings jointly for two feature vectors representing data of different modalities or data from distinct types of entities. Our approach compares favorably against other dimensionality reduction methods, and against a state-of-the-art method of bilinear regression for predicting gene-disease associations.
  arXiv  Detail & Related papers  (2021-01-14T08:48:37Z)
• Slice Sampling for General Completely Random Measures [74.24975039689893]
  We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables. The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
  arXiv  Detail & Related papers  (2020-06-24T17:53:53Z)
• A Semiparametric Approach to Interpretable Machine Learning [9.87381939016363]
  Black box models in machine learning have demonstrated excellent predictive performance in complex problems and high-dimensional settings. Their lack of transparency and interpretability restrict the applicability of such models in critical decision-making processes. We propose a novel approach to trading off interpretability and performance in prediction models using ideas from semiparametric statistics.
  arXiv  Detail & Related papers  (2020-06-08T16:38:15Z)
• Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks [78.76880041670904]
  In neural networks with binary activations and or binary weights the training by gradient descent is complicated. We propose a new method for this estimation problem combining sampling and analytic approximation steps. We experimentally show higher accuracy in gradient estimation and demonstrate a more stable and better performing training in deep convolutional models.
  arXiv  Detail & Related papers  (2020-06-04T21:51:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.