A third-order finite difference weighted essentially non-oscillatory scheme with shallow neural network

• URL: http://arxiv.org/abs/2407.06333v2
• Date: Wed, 10 Jul 2024 09:43:58 GMT
• Title: A third-order finite difference weighted essentially non-oscillatory scheme with shallow neural network
• Authors: Kwanghyuk Park, Xinjuan Chen, Dongjin Lee, Jiaxi Gu, Jae-Hun Jung,
• Abstract summary: We introduce the finite difference weighted essentially non-oscillatory (WENO) scheme based on the neural network for hyperbolic conservation laws. We employ the supervised learning and design two loss functions, one with the mean squared error and the other with the mean squared logarithmic error, where the WENO3-JS weights are computed as the labels. These constructed WENO3-SNN schemes show the outperformed results in one-dimensional examples and improved behavior in two-dimensional examples, compared with the simulations from WENO3-JS and WENO3-Z.
• Score: 9.652617666391926
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we introduce the finite difference weighted essentially non-oscillatory (WENO) scheme based on the neural network for hyperbolic conservation laws. We employ the supervised learning and design two loss functions, one with the mean squared error and the other with the mean squared logarithmic error, where the WENO3-JS weights are computed as the labels. Each loss function consists of two components where the first component compares the difference between the weights from the neural network and WENO3-JS weights, while the second component matches the output weights of the neural network and the linear weights. The former of the loss function enforces the neural network to follow the WENO properties, implying that there is no need for the post-processing layer. Additionally the latter leads to better performance around discontinuities. As a neural network structure, we choose the shallow neural network (SNN) for computational efficiency with the Delta layer consisting of the normalized undivided differences. These constructed WENO3-SNN schemes show the outperformed results in one-dimensional examples and improved behavior in two-dimensional examples, compared with the simulations from WENO3-JS and WENO3-Z.

Related papers

• Parallel-in-Time Solutions with Random Projection Neural Networks [0.07282584715927627]
  This paper considers one of the fundamental parallel-in-time methods for the solution of ordinary differential equations, Parareal, and extends it by adopting a neural network as a coarse propagator. We provide a theoretical analysis of the convergence properties of the proposed algorithm and show its effectiveness for several examples, including Lorenz and Burgers' equations.
  arXiv  Detail & Related papers  (2024-08-19T07:32:41Z)
• LinSATNet: The Positive Linear Satisfiability Neural Networks [116.65291739666303]
  This paper studies how to introduce the popular positive linear satisfiability to neural networks. We propose the first differentiable satisfiability layer based on an extension of the classic Sinkhorn algorithm for jointly encoding multiple sets of marginal distributions.
  arXiv  Detail & Related papers  (2024-07-18T22:05:21Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Enhancing Convergence Speed with Feature-Enforcing Physics-Informed Neural Networks: Utilizing Boundary Conditions as Prior Knowledge for Faster Convergence [0.0]
  This study introduces an accelerated training method for Vanilla Physics-Informed-Neural-Networks (PINN) It addresses three factors that imbalance the loss function: initial weight state of a neural network, domain to boundary points ratio, and loss weighting factor. It is found that incorporating weights generated in the first training phase into the structure of a neural network neutralizes the effects of imbalance factors.
  arXiv  Detail & Related papers  (2023-08-17T09:10:07Z)
• Nonparametric regression using over-parameterized shallow ReLU neural networks [10.339057554827392]
  We show that neural networks can achieve minimax optimal rates of convergence (up to logarithmic factors) for learning functions from certain smooth function classes. It is assumed that the regression function is from the H"older space with smoothness $alpha(d+3)/2$ or a variation space corresponding to shallow neural networks. As a byproduct, we derive a new size-independent bound for the local Rademacher complexity of shallow ReLU neural networks.
  arXiv  Detail & Related papers  (2023-06-14T07:42:37Z)
• Neural Functional Transformers [99.98750156515437]
  This paper uses the attention mechanism to define a novel set of permutation equivariant weight-space layers called neural functional Transformers (NFTs) NFTs respect weight-space permutation symmetries while incorporating the advantages of attention, which have exhibited remarkable success across multiple domains. We also leverage NFTs to develop Inr2Array, a novel method for computing permutation invariant representations from the weights of implicit neural representations (INRs)
  arXiv  Detail & Related papers  (2023-05-22T23:38:27Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks [91.3755431537592]
  We study how random pruning of the weights affects a neural network's neural kernel (NTK) In particular, this work establishes an equivalence of the NTKs between a fully-connected neural network and its randomly pruned version.
  arXiv  Detail & Related papers  (2022-03-27T15:22:19Z)
• Reduced Order Modeling using Shallow ReLU Networks with Grassmann Layers [0.0]
  This paper presents a nonlinear model reduction method for systems of equations using a structured neural network. We show that our method can be applied to scientific problems in the data-scarce regime, which is typically not well-suited for neural network approximations.
  arXiv  Detail & Related papers  (2020-12-17T21:35:06Z)
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism [1.6114012813668932]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising application of deep neural networks to the numerical solution of nonlinear partial differential equations (PDEs) We propose a fundamentally new way to train PINNs adaptively, where the adaptation weights are fully trainable and applied to each training point individually. In numerical experiments with several linear and nonlinear benchmark problems, the SA-PINN outperformed other state-of-the-art PINN algorithm in L2 error.
  arXiv  Detail & Related papers  (2020-09-07T04:07:52Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.