De Rham compatible Deep Neural Network FEM
- URL: http://arxiv.org/abs/2201.05395v3
- Date: Fri, 2 Jun 2023 08:38:19 GMT
- Title: De Rham compatible Deep Neural Network FEM
- Authors: Marcello Longo, Joost A. A. Opschoor, Nico Disch, Christoph Schwab,
Jakob Zech
- Abstract summary: We generalize previous results in that no restrictions on the regular simplicial partitions $mathcalT$ of $Omega$ are required for DNN.
We indicate generalizations of our constructions to higher-order compatible spaces and other, non-compatible classes of discretizations.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: On general regular simplicial partitions $\mathcal{T}$ of bounded polytopal
domains $\Omega \subset \mathbb{R}^d$, $d\in\{2,3\}$, we construct \emph{exact
neural network (NN) emulations} of all lowest order finite element spaces in
the discrete de Rham complex. These include the spaces of piecewise constant
functions, continuous piecewise linear (CPwL) functions, the classical
``Raviart-Thomas element'', and the ``N\'{e}d\'{e}lec edge element''. For all
but the CPwL case, our network architectures employ both ReLU (rectified linear
unit) and BiSU (binary step unit) activations to capture discontinuities. In
the important case of CPwL functions, we prove that it suffices to work with
pure ReLU nets. Our construction and DNN architecture generalizes previous
results in that no geometric restrictions on the regular simplicial partitions
$\mathcal{T}$ of $\Omega$ are required for DNN emulation. In addition, for CPwL
functions our DNN construction is valid in any dimension $d\geq 2$. Our
``FE-Nets'' are required in the variationally correct, structure-preserving
approximation of boundary value problems of electromagnetism in nonconvex
polyhedra $\Omega \subset \mathbb{R}^3$. They are thus an essential ingredient
in the application of e.g., the methodology of ``physics-informed NNs'' or
``deep Ritz methods'' to electromagnetic field simulation via deep learning
techniques. We indicate generalizations of our constructions to higher-order
compatible spaces and other, non-compatible classes of discretizations, in
particular the ``Crouzeix-Raviart'' elements and Hybridized, Higher Order (HHO)
methods.
Related papers
- Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks.
In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
arXiv Detail & Related papers (2024-04-29T15:04:07Z) - Exponential Expressivity of ReLU$^k$ Neural Networks on Gevrey Classes with Point Singularities [0.0]
We prove exponential emulation rates in Sobolev spaces in terms of the number of neurons.
On shape-regular, simplicial partitions of polytopal domains $mathrmD$, both the number of neurons and the number of nonzero parameters are proportional to the number of degrees of freedom of the finite element space.
arXiv Detail & Related papers (2024-03-04T13:39:22Z) - Efficiently Learning One-Hidden-Layer ReLU Networks via Schur
Polynomials [50.90125395570797]
We study the problem of PAC learning a linear combination of $k$ ReLU activations under the standard Gaussian distribution on $mathbbRd$ with respect to the square loss.
Our main result is an efficient algorithm for this learning task with sample and computational complexity $(dk/epsilon)O(k)$, whereepsilon>0$ is the target accuracy.
arXiv Detail & Related papers (2023-07-24T14:37:22Z) - Polynomial Width is Sufficient for Set Representation with
High-dimensional Features [69.65698500919869]
DeepSets is the most widely used neural network architecture for set representation.
We present two set-element embedding layers: (a) linear + power activation (LP) and (b) linear + exponential activations (LE)
arXiv Detail & Related papers (2023-07-08T16:00:59Z) - Quantum and classical low-degree learning via a dimension-free Remez
inequality [52.12931955662553]
We show a new way to relate functions on the hypergrid to their harmonic extensions over the polytorus.
We show the supremum of a function $f$ over products of the cyclic group $exp(2pi i k/K)_k=1K$.
We extend to new spaces a recent line of work citeEI22, CHP, VZ22 that gave similarly efficient methods for learning low-degrees on hypercubes and observables on qubits.
arXiv Detail & Related papers (2023-01-04T04:15:40Z) - Minimal Width for Universal Property of Deep RNN [6.744583770038476]
A recurrent neural network (RNN) is a widely used deep-learning network for dealing with sequential data.
We prove the universality of deep narrow RNNs and show that the upper bound of the minimum width for universality can be independent of the length of the data.
arXiv Detail & Related papers (2022-11-25T02:43:54Z) - Average-Case Complexity of Tensor Decomposition for Low-Degree
Polynomials [93.59919600451487]
"Statistical-computational gaps" occur in many statistical inference tasks.
We consider a model for random order-3 decomposition where one component is slightly larger in norm than the rest.
We show that tensor entries can accurately estimate the largest component when $ll n3/2$ but fail to do so when $rgg n3/2$.
arXiv Detail & Related papers (2022-11-10T00:40:37Z) - Neural Networks Efficiently Learn Low-Dimensional Representations with
SGD [22.703825902761405]
We show that SGD-trained ReLU NNs can learn a single-index target of the form $y=f(langleboldsymbolu,boldsymbolxrangle) + epsilon$ by recovering the principal direction.
We also provide compress guarantees for NNs using the approximate low-rank structure produced by SGD.
arXiv Detail & Related papers (2022-09-29T15:29:10Z) - Neural Network Approximation of Continuous Functions in High Dimensions
with Applications to Inverse Problems [6.84380898679299]
Current theory predicts that networks should scale exponentially in the dimension of the problem.
We provide a general method for bounding the complexity required for a neural network to approximate a H"older (or uniformly) continuous function.
arXiv Detail & Related papers (2022-08-28T22:44:07Z) - Minimax Optimal Quantization of Linear Models: Information-Theoretic
Limits and Efficient Algorithms [59.724977092582535]
We consider the problem of quantizing a linear model learned from measurements.
We derive an information-theoretic lower bound for the minimax risk under this setting.
We show that our method and upper-bounds can be extended for two-layer ReLU neural networks.
arXiv Detail & Related papers (2022-02-23T02:39:04Z) - Exponential ReLU Neural Network Approximation Rates for Point and Edge
Singularities [0.0]
We prove exponential expressivity with stable ReLU Neural Networks (ReLU NNs) in $H1(Omega)$ for weighted analytic function classes in certain polytopal domains.
The exponential approximation rates are shown to hold in space dimension $d = 2$ on Lipschitz polygons with straight sides, and in space dimension $d=3$ on Fichera-type polyhedral domains with plane faces.
arXiv Detail & Related papers (2020-10-23T07:44:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.