Model Averaging and Double Machine Learning
- URL: http://arxiv.org/abs/2401.01645v1
- Date: Wed, 3 Jan 2024 09:38:13 GMT
- Title: Model Averaging and Double Machine Learning
- Authors: Achim Ahrens and Christian B. Hansen and Mark E. Schaffer and Thomas
Wiemann
- Abstract summary: We introduce two new stacking approaches for DDML: short-stacking exploits the cross-fitting step and pooled stacking enforces common stacking weights over cross-fitting folds.
We show that DDML with stacking is more robust to partially unknown functional forms than common alternative approaches based on single pre-selected learners.
- Score: 2.8880000014100506
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper discusses pairing double/debiased machine learning (DDML) with
stacking, a model averaging method for combining multiple candidate learners,
to estimate structural parameters. We introduce two new stacking approaches for
DDML: short-stacking exploits the cross-fitting step of DDML to substantially
reduce the computational burden and pooled stacking enforces common stacking
weights over cross-fitting folds. Using calibrated simulation studies and two
applications estimating gender gaps in citations and wages, we show that DDML
with stacking is more robust to partially unknown functional forms than common
alternative approaches based on single pre-selected learners. We provide Stata
and R software implementing our proposals.
Related papers
- MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation [80.47072100963017]
Model merging is an effective approach to combine multiple single-task models, fine-tuned from the same pre-trained model, into a multitask model.
Existing model-merging methods focus on enhancing average task accuracy.
We introduce a novel low-compute algorithm, Model Merging with Amortized Pareto Front (MAP)
arXiv Detail & Related papers (2024-06-11T17:55:25Z) - Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable.
CMDPs serve as an important framework to model many real-world applications with time-varying environments.
We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
arXiv Detail & Related papers (2024-02-05T03:25:04Z) - Online Multi-Task Learning with Recursive Least Squares and Recursive Kernel Methods [50.67996219968513]
We introduce two novel approaches for Online Multi-Task Learning (MTL) Regression Problems.
We achieve exact and approximate recursions with quadratic per-instance cost on the dimension of the input space.
We compare our online MTL methods to other contenders in a real-world wind speed forecasting case study.
arXiv Detail & Related papers (2023-08-03T01:41:34Z) - ddml: Double/debiased machine learning in Stata [2.8880000014100506]
We introduce the package ddml for Double/Debiased Machine Learning (DDML) in Stata.
ddml is compatible with many existing supervised machine learning programs in Stata.
arXiv Detail & Related papers (2023-01-23T12:37:34Z) - DoubleML -- An Object-Oriented Implementation of Double Machine Learning in R [4.830430752756141]
R package DoubleML implements the double/debiased machine learning framework.
It provides functionalities to estimate parameters in causal models based on machine learning methods.
arXiv Detail & Related papers (2021-03-17T12:42:41Z) - Contrastive Prototype Learning with Augmented Embeddings for Few-Shot
Learning [58.2091760793799]
We propose a novel contrastive prototype learning with augmented embeddings (CPLAE) model.
With a class prototype as an anchor, CPL aims to pull the query samples of the same class closer and those of different classes further away.
Extensive experiments on several benchmarks demonstrate that our proposed CPLAE achieves new state-of-the-art.
arXiv Detail & Related papers (2021-01-23T13:22:44Z) - An Online Method for A Class of Distributionally Robust Optimization
with Non-Convex Objectives [54.29001037565384]
We propose a practical online method for solving a class of online distributionally robust optimization (DRO) problems.
Our studies demonstrate important applications in machine learning for improving the robustness of networks.
arXiv Detail & Related papers (2020-06-17T20:19:25Z) - Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method.
PCL implicitly encodes semantic structures of the data into the learned embedding space.
PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
arXiv Detail & Related papers (2020-05-11T09:53:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.