Instrument Space Selection for Kernel Maximum Moment Restriction

• URL: http://arxiv.org/abs/2106.03340v1
• Date: Mon, 7 Jun 2021 05:07:12 GMT
• Title: Instrument Space Selection for Kernel Maximum Moment Restriction
• Authors: Rui Zhang, Krikamol Muandet, Bernhard Sch\"olkopf, Masaaki Imaizumi
• Abstract summary: We present a systematic way to select the instrument space for parameter estimation based on a principle of the least identifiable instrument space (LIIS) Our selection criterion combines two distinct objectives to determine such an optimal space: (i) a test criterion to check identifiability; (ii) an information criterion based on the effective dimension of RKHSs as a complexity measure.
• Score: 20.343786646988477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Kernel maximum moment restriction (KMMR) recently emerges as a popular framework for instrumental variable (IV) based conditional moment restriction (CMR) models with important applications in conditional moment (CM) testing and parameter estimation for IV regression and proximal causal learning. The effectiveness of this framework, however, depends critically on the choice of a reproducing kernel Hilbert space (RKHS) chosen as a space of instruments. In this work, we presents a systematic way to select the instrument space for parameter estimation based on a principle of the least identifiable instrument space (LIIS) that identifies model parameters with the least space complexity. Our selection criterion combines two distinct objectives to determine such an optimal space: (i) a test criterion to check identifiability; (ii) an information criterion based on the effective dimension of RKHSs as a complexity measure. We analyze the consistency of our method in determining the LIIS, and demonstrate its effectiveness for parameter estimation via simulations.

Related papers

• Stochastic Optimization with Optimal Importance Sampling [49.484190237840714]
  We propose an iterative-based algorithm that jointly updates the decision and the IS distribution without requiring time-scale separation between the two. Our method achieves the lowest possible variable variance and guarantees global convergence under convexity of the objective and mild assumptions on the IS distribution family.
  arXiv  Detail & Related papers  (2025-04-04T16:10:18Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective. The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning. The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection [55.2480439325792]
  This paper presents a hybrid framework that integrates both statistical feature selection and classification techniques to improve defect detection accuracy. We present around 55 distinguished features that are extracted from industrial images, which are then analyzed using statistical methods. By integrating these methods with flexible machine learning applications, the proposed framework improves detection accuracy and reduces false positives and misclassifications.
  arXiv  Detail & Related papers  (2024-12-11T22:12:21Z)
• Statistical Inference for Temporal Difference Learning with Linear Function Approximation [62.69448336714418]
  Temporal Difference (TD) learning, arguably the most widely used for policy evaluation, serves as a natural framework for this purpose. In this paper, we study the consistency properties of TD learning with Polyak-Ruppert averaging and linear function approximation, and obtain three significant improvements over existing results.
  arXiv  Detail & Related papers  (2024-10-21T15:34:44Z)
• Minimax Optimal Goodness-of-Fit Testing with Kernel Stein Discrepancy [13.429541377715298]
  We explore the minimax optimality of goodness-of-fit tests on general domains using the kernelized Stein discrepancy (KSD) The KSD framework offers a flexible approach for goodness-of-fit testing, avoiding strong distributional assumptions. We introduce an adaptive test capable of achieving minimax optimality up to a logarithmic factor by adapting to unknown parameters.
  arXiv  Detail & Related papers  (2024-04-12T07:06:12Z)
• Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
  We propose an effective method called Latent Semantic Consensus (LSC) LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses. LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
  arXiv  Detail & Related papers  (2024-03-11T05:35:38Z)
• Support Vector Machine for Determining Euler Angles in an Inertial Navigation System [55.41644538483948]
  The paper discusses the improvement of the accuracy of an inertial navigation system created on the basis of MEMS sensors using machine learning (ML) methods. The proposed algorithm based on MO has demonstrated its ability to correctly classify in the presence of noise typical for MEMS sensors.
  arXiv  Detail & Related papers  (2022-12-07T10:01:11Z)
• A Theoretical Framework of Almost Hyperparameter-free Hyperparameter Selection Methods for Offline Policy Evaluation [2.741266294612776]
  offline reinforcement learning (OPE) is a core technology for data-driven decision optimization without environment simulators. We introduce a new approximate hyper parameter selection framework for OPE, which defines a notion of optimality (called selection criteria) in a quantitative and interpretable manner. We derive four AHS methods each of which has different characteristics such as convergence rate and time complexity.
  arXiv  Detail & Related papers  (2022-01-07T02:23:09Z)
• Sinkhorn Natural Gradient for Generative Models [125.89871274202439]
  We propose a novel Sinkhorn Natural Gradient (SiNG) algorithm which acts as a steepest descent method on the probability space endowed with the Sinkhorn divergence. We show that the Sinkhorn information matrix (SIM), a key component of SiNG, has an explicit expression and can be evaluated accurately in complexity that scales logarithmically. In our experiments, we quantitatively compare SiNG with state-of-the-art SGD-type solvers on generative tasks to demonstrate its efficiency and efficacy of our method.
  arXiv  Detail & Related papers  (2020-11-09T02:51:17Z)
• A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces [53.47210316424326]
  KeRNS is an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes. We prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time.
  arXiv  Detail & Related papers  (2020-07-09T21:37:13Z)
• On the minmax regret for statistical manifolds: the role of curvature [68.8204255655161]
  Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models. We derive a sharper expression than the standard one given by the complexity, where the scalar curvature of the Fisher information metric plays a dominant role.
  arXiv  Detail & Related papers  (2020-07-06T17:28:19Z)
• Parameterized MDPs and Reinforcement Learning Problems -- A Maximum Entropy Principle Based Framework [2.741266294612776]
  We present a framework to address a class of sequential decision making problems. Our framework features learning the optimal control policy with robustness to noisy data.
  arXiv  Detail & Related papers  (2020-06-17T04:08:35Z)
• Minimax Estimation of Conditional Moment Models [40.95498063465325]
  We introduce a min-max criterion function, under which the estimation problem can be thought of as solving a zero-sum game. We analyze the statistical estimation rate of the resulting estimator for arbitrary hypothesis spaces. We show how our modified mean squared error rate, combined with conditions that bound the ill-posedness of the inverse problem, lead to mean squared error rates.
  arXiv  Detail & Related papers  (2020-06-12T14:02:38Z)
• Selective machine learning of doubly robust functionals [6.880360838661036]
  We propose a selective machine learning framework for making inferences about a finite-dimensional functional defined on a semiparametric model. We introduce a new selection criterion aimed at bias reduction in estimating the functional of interest based on a novel definition of pseudo-risk.
  arXiv  Detail & Related papers  (2019-11-05T19:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.