DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation

• URL: http://arxiv.org/abs/2408.02045v1
• Date: Sun, 4 Aug 2024 14:45:26 GMT
• Title: DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation
• Authors: Qinshuo Liu, Zixin Wang, Xi-An Li, Xinyao Ji, Lei Zhang, Lin Liu, Zhonghua Liu,
• Abstract summary: Semiparametric statistics play a pivotal role in a wide range of domains, including but not limited to missing data, causal inference, and transfer learning. We develop a scalable algorithm called Deep Neural-Nets Assisted Semiparametric Estimation (DNA-SE) by leveraging the universal approximation property of Deep Neural-Nets (DNN) to streamline semiparametric procedures.
• Score: 39.48526221316346
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semiparametric statistics play a pivotal role in a wide range of domains, including but not limited to missing data, causal inference, and transfer learning, to name a few. In many settings, semiparametric theory leads to (nearly) statistically optimal procedures that yet involve numerically solving Fredholm integral equations of the second kind. Traditional numerical methods, such as polynomial or spline approximations, are difficult to scale to multi-dimensional problems. Alternatively, statisticians may choose to approximate the original integral equations by ones with closed-form solutions, resulting in computationally more efficient, but statistically suboptimal or even incorrect procedures. To bridge this gap, we propose a novel framework by formulating the semiparametric estimation problem as a bi-level optimization problem; and then we develop a scalable algorithm called Deep Neural-Nets Assisted Semiparametric Estimation (DNA-SE) by leveraging the universal approximation property of Deep Neural-Nets (DNN) to streamline semiparametric procedures. Through extensive numerical experiments and a real data analysis, we demonstrate the numerical and statistical advantages of $\dnase$ over traditional methods. To the best of our knowledge, we are the first to bring DNN into semiparametric statistics as a numerical solver of integral equations in our proposed general framework.

Related papers

• Solving Poisson Equations using Neural Walk-on-Spheres [80.1675792181381]
  We propose Neural Walk-on-Spheres (NWoS), a novel neural PDE solver for the efficient solution of high-dimensional Poisson equations. We demonstrate the superiority of NWoS in accuracy, speed, and computational costs.
  arXiv  Detail & Related papers  (2024-06-05T17:59:22Z)
• Spectral operator learning for parametric PDEs without data reliance [6.7083321695379885]
  We introduce a novel operator learning-based approach for solving parametric partial differential equations (PDEs) without the need for data harnessing. The proposed framework demonstrates superior performance compared to existing scientific machine learning techniques.
  arXiv  Detail & Related papers  (2023-10-03T12:37:15Z)
• A multiobjective continuation method to compute the regularization path of deep neural networks [1.3654846342364308]
  Sparsity is a highly feature in deep neural networks (DNNs) since it ensures numerical efficiency, improves the interpretability of models, and robustness. We present an algorithm that allows for the entire sparse front for the above-mentioned objectives in a very efficient manner for high-dimensional gradients with millions of parameters. We demonstrate that knowledge of the regularization path allows for a well-generalizing network parametrization.
  arXiv  Detail & Related papers  (2023-08-23T10:08:52Z)
• Multilevel CNNs for Parametric PDEs [0.0]
  We combine concepts from multilevel solvers for partial differential equations with neural network based deep learning. An in-depth theoretical analysis shows that the proposed architecture is able to approximate multigrid V-cycles to arbitrary precision. We find substantial improvements over state-of-the-art deep learning-based solvers.
  arXiv  Detail & Related papers  (2023-04-01T21:11:05Z)
• FaDIn: Fast Discretized Inference for Hawkes Processes with General Parametric Kernels [82.53569355337586]
  This work offers an efficient solution to temporal point processes inference using general parametric kernels with finite support. The method's effectiveness is evaluated by modeling the occurrence of stimuli-induced patterns from brain signals recorded with magnetoencephalography (MEG) Results show that the proposed approach leads to an improved estimation of pattern latency than the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-10T12:35:02Z)
• Improving Computational Complexity in Statistical Models with Second-Order Information [32.64382185990981]
  We study the normalized gradient descent (NormGD) algorithm for solving parameter estimation in parametric statistical models. We demonstrate that the NormGD algorithm achieves the optimal overall computational complexity $mathcalO(n)$ to reach the final statistical radius. This computational complexity is cheaper than that of the fixed step-size gradient descent algorithm.
  arXiv  Detail & Related papers  (2022-02-09T01:32:50Z)
• Linear regression with partially mismatched data: local search with theoretical guarantees [9.398989897176953]
  We study an important variant of linear regression in which the predictor-response pairs are partially mismatched. We use an optimization formulation to simultaneously learn the underlying regression coefficients and the permutation corresponding to the mismatches. We prove that our local search algorithm converges to a nearly-optimal solution at a linear rate.
  arXiv  Detail & Related papers  (2021-06-03T23:32:12Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.