Is completeness necessary? Estimation in nonidentified linear models

• URL: http://arxiv.org/abs/1709.03473v5
• Date: Mon, 06 Jan 2025 17:09:35 GMT
• Title: Is completeness necessary? Estimation in nonidentified linear models
• Authors: Andrii Babii, Jean-Pierre Florens,
• Abstract summary: We develop a theory of regularized estimators, which include methods such as high-dimensional ridge regularization, gradient descent, and principal component analysis (PCA) The results are illustrated for high-dimensional and nonparametric instrumental variable regressions and are supported through simulation experiments.
• Score: 0.0
• License:
• Abstract: Modern data analysis depends increasingly on estimating models via flexible high-dimensional or nonparametric machine learning methods, where the identification of structural parameters is often challenging and untestable. In linear settings, this identification hinges on the completeness condition, which requires the nonsingularity of a high-dimensional matrix or operator and may fail for finite samples or even at the population level. Regularized estimators provide a solution by enabling consistent estimation of structural or average structural functions, sometimes even under identification failure. We show that the asymptotic distribution in these cases can be nonstandard. We develop a comprehensive theory of regularized estimators, which include methods such as high-dimensional ridge regularization, gradient descent, and principal component analysis (PCA). The results are illustrated for high-dimensional and nonparametric instrumental variable regressions and are supported through simulation experiments.

Related papers

• Fast and Reliable Probabilistic Reflectometry Inversion with Prior-Amortized Neural Posterior Estimation [73.81105275628751]
  Finding all structures compatible with reflectometry data is computationally prohibitive for standard algorithms. We address this lack of reliability with a probabilistic deep learning method that identifies all realistic structures in seconds. Our method, Prior-Amortized Neural Posterior Estimation (PANPE), combines simulation-based inference with novel adaptive priors.
  arXiv  Detail & Related papers  (2024-07-26T10:29:16Z)
• On the Detection and Quantification of Nonlinearity via Statistics of the Gradients of a Black-Box Model [0.0]
  Detection and identification of nonlinearity is a task of high importance for structural dynamics. A method to detect nonlinearity is proposed, based on the distribution of the gradients of a data-driven model.
  arXiv  Detail & Related papers  (2023-02-15T23:15:22Z)
• Linear Stability Hypothesis and Rank Stratification for Nonlinear Models [3.0041514772139166]
  We propose a rank stratification for general nonlinear models to uncover a model rank as an "effective size of parameters" By these results, model rank of a target function predicts a minimal training data size for its successful recovery.
  arXiv  Detail & Related papers  (2022-11-21T16:27:25Z)
• Identifiability and Asymptotics in Learning Homogeneous Linear ODE Systems from Discrete Observations [114.17826109037048]
  Ordinary Differential Equations (ODEs) have recently gained a lot of attention in machine learning. theoretical aspects, e.g., identifiability and properties of statistical estimation are still obscure. This paper derives a sufficient condition for the identifiability of homogeneous linear ODE systems from a sequence of equally-spaced error-free observations sampled from a single trajectory.
  arXiv  Detail & Related papers  (2022-10-12T06:46:38Z)
• NeuralSI: Structural Parameter Identification in Nonlinear Dynamical Systems [9.77270939559057]
  This paper explores a new framework, dubbed NeuralSI, for structural identification. Our approach seeks to estimate nonlinear parameters from governing equations. The trained model can also be extrapolated under both standard and extreme conditions.
  arXiv  Detail & Related papers  (2022-08-26T16:32:51Z)
• A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization [1.2882319878552302]
  This paper develops a new iterative learning algorithm for complex turbulent systems with partial observations. It alternates between identifying model structures, recovering unobserved variables, and estimating parameters. Numerical experiments show that the new algorithm succeeds in identifying the model structure and providing suitable parameterizations for many complex nonlinear systems.
  arXiv  Detail & Related papers  (2022-08-19T00:35:03Z)
• Function Classes for Identifiable Nonlinear Independent Component Analysis [10.828616610785524]
  Unsupervised learning of latent variable models (LVMs) is widely used to represent data in machine learning. Recent work suggests that constraining the function class of such models may promote identifiability. We prove that a subclass of these transformations, conformal maps, is identifiable and provide novel theoretical results.
  arXiv  Detail & Related papers  (2022-08-12T17:58:31Z)
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows [58.720142291102135]
  We use isometries to separate manifold learning and density estimation. We also employ autoencoders to design embeddings with explicit inverses that do not distort the probability distribution.
  arXiv  Detail & Related papers  (2022-03-08T08:57:43Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.