Approximation of Functionals by Neural Network without Curse of Dimensionality

• URL: http://arxiv.org/abs/2205.14421v1
• Date: Sat, 28 May 2022 13:09:43 GMT
• Title: Approximation of Functionals by Neural Network without Curse of Dimensionality
• Abstract summary: In this paper, we establish a neural network to approximate functionals. The key idea of the approximation is to define a Barron space of functionals.
• Score: 12.717839871971517
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we establish a neural network to approximate functionals, which are maps from infinite dimensional spaces to finite dimensional spaces. The approximation error of the neural network is $O(1/\sqrt{m})$ where $m$ is the size of networks, which overcomes the curse of dimensionality. The key idea of the approximation is to define a Barron space of functionals.

Related papers

• Dimension-independent learning rates for high-dimensional classification problems [53.622581586464634]
  We show that every $RBV2$ function can be approximated by a neural network with bounded weights. We then prove the existence of a neural network with bounded weights approximating a classification function.
  arXiv  Detail & Related papers  (2024-09-26T16:02:13Z)
• Optimal Neural Network Approximation for High-Dimensional Continuous Functions [5.748690310135373]
  We present a family of continuous functions that requires at least width $d$, and therefore at least $d$ intrinsic neurons, to achieve arbitrary accuracy in its approximation. This shows that the requirement of $mathcalO(d)$ intrinsic neurons is optimal in the sense that it grows linearly with the input dimension $d$.
  arXiv  Detail & Related papers  (2024-09-04T01:18:55Z)
• Towards an Algebraic Framework For Approximating Functions Using Neural Network Polynomials [0.589889361990138]
  We make the case for neural network objects and extend an already existing neural network calculus explained in detail in Chapter 2 on citebigbook. Our aim will be to show that, yes, indeed, it makes sense to talk about neural networks, neural network exponentials, sine, and cosines in the sense that they do indeed approximate their real number counterparts subject to limitations on certain parameters, $q$, and $varepsilon$.
  arXiv  Detail & Related papers  (2024-02-01T23:06:50Z)
• Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds [75.51968172401394]
  We study the sample complexity of the neural policy mirror descent (NPMD) algorithm with deep convolutional neural networks (CNN) In each iteration of NPMD, both the value function and the policy can be well approximated by CNNs. We show that NPMD can leverage the low-dimensional structure of state space to escape from the curse of dimensionality.
  arXiv  Detail & Related papers  (2023-09-25T07:31:22Z)
• Piecewise Linear Functions Representable with Infinite Width Shallow ReLU Neural Networks [0.0]
  We prove a conjecture of Ongie et al. that every continuous piecewise linear function expressible with this kind of infinite width neural network is expressible as a finite width shallow ReLU neural network.
  arXiv  Detail & Related papers  (2023-07-25T15:38:18Z)
• Activation Functions Not To Active: A Plausible Theory on Interpreting Neural Networks [0.0]
  Researchers believe that neural networks model a high-dimensional space but cannot give a clear definition of this space. We develop a plausible theory on interpreting neural networks in terms of the role of activation functions in neural networks. We show that the activation function acts as a magnifying function that maps the low-dimensional linear space into an infinite-dimensional space.
  arXiv  Detail & Related papers  (2023-05-01T05:23:58Z)
• Deep neural network expressivity for optimal stopping problems [2.741266294612776]
  An optimal stopping problem can be approximated with error at most $varepsilon$ by a deep ReLU neural network of size at most $kappa dmathfrakq varepsilon-mathfrakr$. This proves that deep neural networks do not suffer from the curse of dimensionality when employed to solve optimal stopping problems.
  arXiv  Detail & Related papers  (2022-10-19T10:22:35Z)
• Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons [5.37133760455631]
  We develop feed-forward neural networks that achieve the universal approximation property for all continuous functions with a fixed finite number of neurons. We prove that $sigma$-activated networks with width $36d(2d+1)$ and depth $11$ can approximate any continuous function on a $d$-dimensioanl hypercube within an arbitrarily small error.
  arXiv  Detail & Related papers  (2021-07-06T05:24:30Z)
• Deep neural network approximation of analytic functions [91.3755431537592]
  entropy bound for the spaces of neural networks with piecewise linear activation functions. We derive an oracle inequality for the expected error of the considered penalized deep neural network estimators.
  arXiv  Detail & Related papers  (2021-04-05T18:02:04Z)
• Size and Depth Separation in Approximating Natural Functions with Neural Networks [52.73592689730044]
  We show the benefits of size and depth for approximation of natural functions with ReLU networks. We show a complexity-theoretic barrier to proving such results beyond size $O(d)$. We also show an explicit natural function, that can be approximated with networks of size $O(d)$.
  arXiv  Detail & Related papers  (2021-01-30T21:30:11Z)
• A deep network construction that adapts to intrinsic dimensionality beyond the domain [79.23797234241471]
  We study the approximation of two-layer compositions $f(x) = g(phi(x))$ via deep networks with ReLU activation. We focus on two intuitive and practically relevant choices for $phi$: the projection onto a low-dimensional embedded submanifold and a distance to a collection of low-dimensional sets.
  arXiv  Detail & Related papers  (2020-08-06T09:50:29Z)
• Neural Operator: Graph Kernel Network for Partial Differential Equations [57.90284928158383]
  This work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators) We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.
  arXiv  Detail & Related papers  (2020-03-07T01:56:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.