Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs

• URL: http://arxiv.org/abs/2507.01533v1
• Date: Wed, 02 Jul 2025 09:37:16 GMT
• Title: Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs
• Authors: Hanno Gottschalk, Emil Partow, Tobias J. Riedlinger,
• Abstract summary: This paper provides a proof of the consistency of sparse grid quadrature for numerical integration of high dimensional distributions.<n>A decomposition of the total numerical error in quadrature error and statistical error is provided.
• Score: 1.3654846342364308
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper provides a proof of the consistency of sparse grid quadrature for numerical integration of high dimensional distributions. In a first step, a transport map is learned that normalizes the distribution to a noise distribution on the unit cube. This step is built on the statistical learning theory of neural ordinary differential equations, which has been established recently. Secondly, the composition of the generative map with the quantity of interest is integrated numerically using the Clenshaw-Curtis sparse grid quadrature. A decomposition of the total numerical error in quadrature error and statistical error is provided. As main result it is proven in the framework of empirical risk minimization that all error terms can be controlled in the sense of PAC (probably approximately correct) learning and with high probability the numerical integral approximates the theoretical value up to an arbitrary small error in the limit where the data set size is growing and the network capacity is increased adaptively.

Related papers

• Modes of Sequence Models and Learning Coefficients [0.6906005491572401]
  We develop a geometric account of sequence modelling that links patterns in the data to measurable properties of the loss landscape in transformer networks.<n>We show theoretically that Local Learning Coefficient estimates are insensitive to modes below a data-dependent threshold.<n>This insight clarifies why reliable LLC estimates can be obtained even when a network parameter is not a strict minimiser of the population loss.
  arXiv  Detail & Related papers  (2025-04-25T03:38:10Z)
• Numerical and statistical analysis of NeuralODE with Runge-Kutta time integration [1.3654846342364308]
  We give a detailed account on the consistency of Maximum Likelihood based empirical risk minimization for a generic class of target measures.<n>We also give a numerical analysis of the NeuralODE algorithm based on the 2nd order Runge-Kutta (RK) time integration.
  arXiv  Detail & Related papers  (2025-03-13T11:58:18Z)
• Learning and generalization of one-hidden-layer neural networks, going beyond standard Gaussian data [14.379261299138147]
  This paper analyzes the convergence and iterations of a one-hidden-layer neural network when the input features follow the Gaussian mixture model. For the first time, this paper characterizes the impact of the input distributions on the sample and the learning rate.
  arXiv  Detail & Related papers  (2022-07-07T23:27:44Z)
• Learning Distributions by Generative Adversarial Networks: Approximation and Generalization [0.6768558752130311]
  We study how well generative adversarial networks learn from finite samples by analyzing the convergence rates of these models. Our analysis is based on a new inequality oracle that decomposes the estimation error of GAN into the discriminator and generator approximation errors. For generator approximation error, we show that neural network can approximately transform a low-dimensional source distribution to a high-dimensional target distribution.
  arXiv  Detail & Related papers  (2022-05-25T09:26:17Z)
• Robust Estimation for Nonparametric Families via Generative Adversarial Networks [92.64483100338724]
  We provide a framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems. Our work extend these to robust mean estimation, second moment estimation, and robust linear regression. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance.
  arXiv  Detail & Related papers  (2022-02-02T20:11:33Z)
• Partial Counterfactual Identification from Observational and Experimental Data [83.798237968683]
  We develop effective Monte Carlo algorithms to approximate the optimal bounds from an arbitrary combination of observational and experimental data. Our algorithms are validated extensively on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-10-12T02:21:30Z)
• The Separation Capacity of Random Neural Networks [78.25060223808936]
  We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability. We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
  arXiv  Detail & Related papers  (2021-07-31T10:25:26Z)
• Predicting Unreliable Predictions by Shattering a Neural Network [145.3823991041987]
  Piecewise linear neural networks can be split into subfunctions. Subfunctions have their own activation pattern, domain, and empirical error. Empirical error for the full network can be written as an expectation over subfunctions.
  arXiv  Detail & Related papers  (2021-06-15T18:34:41Z)
• An error analysis of generative adversarial networks for learning distributions [11.842861158282265]
  generative adversarial networks (GANs) learn probability distributions from finite samples. GANs are able to adaptively learn data distributions with low-dimensional structure or have H"older densities. Our analysis is based on a new oracle inequality decomposing the estimation error into generator and discriminator approximation error and statistical error.
  arXiv  Detail & Related papers  (2021-05-27T08:55:19Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• Log-Likelihood Ratio Minimizing Flows: Towards Robust and Quantifiable Neural Distribution Alignment [52.02794488304448]
  We propose a new distribution alignment method based on a log-likelihood ratio statistic and normalizing flows. We experimentally verify that minimizing the resulting objective results in domain alignment that preserves the local structure of input domains.
  arXiv  Detail & Related papers  (2020-03-26T22:10:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.