Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

• URL: http://arxiv.org/abs/2204.11375v1
• Date: Sun, 24 Apr 2022 22:39:06 GMT
• Title: Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine
• Authors: Naxian Ni, Suchuan Dong
• Abstract summary: The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs) ELM method requires the last hidden layer of the neural network to be wide to achieve a high accuracy. We present a modified ELM method, termed HLConcELM (hidden-layerd ELM), to overcome the above drawback.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we present a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), to overcome the above drawback of the conventional ELM method. The HLConcELM method can produce highly accurate solutions to linear/nonlinear PDEs when the last hidden layer of the network is narrow and when it is wide. The new method is based on a type of modified feedforward neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes. We show that HLConcFNNs have the remarkable property that, given a network architecture, when additional hidden layers are appended to the network or when extra nodes are added to the existing hidden layers, the approximation capacity of the HLConcFNN associated with the new architecture is guaranteed to be not smaller than that of the original network architecture. We present ample benchmark tests with linear/nonlinear PDEs to demonstrate the computational accuracy and performance of the HLConcELM method and the superiority of this method to the conventional ELM from previous works.

Related papers

• LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layers [0.0468732641979009]
  We propose a layer-wise parameterization for convolutional neural networks (CNNs) that includes built-in robustness guarantees. Our method Lip Kernel directly parameterizes dissipative convolution kernels using a 2-D Roesser-type state space model. We show that the run-time using our method is orders of magnitude faster than state-of-the-art Lipschitz-bounded networks.
  arXiv  Detail & Related papers  (2024-10-29T17:20:14Z)
• A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problems [0.0]
  Extreme learning machine (ELM) is a methodology for solving partial differential equations (PDEs) using a single hidden layer feed-forward neural network. In this paper, we propose a nonoverlapping domain decomposition method (DDM) for ELMs that not only reduces the training time of ELMs, but is also suitable for parallel computation.
  arXiv  Detail & Related papers  (2024-06-22T23:25:54Z)
• Adaptive Multilevel Neural Networks for Parametric PDEs with Error Estimation [0.0]
  A neural network architecture is presented to solve high-dimensional parameter-dependent partial differential equations (pPDEs) It is constructed to map parameters of the model data to corresponding finite element solutions. It outputs a coarse grid solution and a series of corrections as produced in an adaptive finite element method (AFEM)
  arXiv  Detail & Related papers  (2024-03-19T11:34:40Z)
• Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding [57.71603937699949]
  We study optimization guarantees, i.e., achieving near-zero training loss with the increase in the number of learning epochs. We show that the threshold on the number of training samples increases with the increase in the network width.
  arXiv  Detail & Related papers  (2023-09-12T13:03:47Z)
• A Unified Algebraic Perspective on Lipschitz Neural Networks [88.14073994459586]
  This paper introduces a novel perspective unifying various types of 1-Lipschitz neural networks. We show that many existing techniques can be derived and generalized via finding analytical solutions of a common semidefinite programming (SDP) condition. Our approach, called SDP-based Lipschitz Layers (SLL), allows us to design non-trivial yet efficient generalization of convex potential layers.
  arXiv  Detail & Related papers  (2023-03-06T14:31:09Z)
• Efficient Decompositional Rule Extraction for Deep Neural Networks [5.69361786082969]
  ECLAIRE is a novel-time rule extraction algorithm capable of scaling to both large DNN architectures and large training datasets. We show that ECLAIRE consistently extracts more accurate and comprehensible rule sets than the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-24T16:54:10Z)
• A novel Deep Neural Network architecture for non-linear system identification [78.69776924618505]
  We present a novel Deep Neural Network (DNN) architecture for non-linear system identification. Inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function) This architecture allows for automatic complexity selection based solely on available data.
  arXiv  Detail & Related papers  (2021-06-06T10:06:07Z)
• A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations [69.73803123972297]
  We propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems.
  arXiv  Detail & Related papers  (2020-12-21T17:39:51Z)
• Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations [0.0]
  We present a neural network-based method for solving linear and nonlinear partial differential equations. The method combines the ideas of extreme learning machines (ELM), domain decomposition and local neural networks. We compare the current method with the deep Galerkin method (DGM) and the physics-informed neural network (PINN) in terms of the accuracy and computational cost.
  arXiv  Detail & Related papers  (2020-12-04T23:19:39Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• MSE-Optimal Neural Network Initialization via Layer Fusion [68.72356718879428]
  Deep neural networks achieve state-of-the-art performance for a range of classification and inference tasks. The use of gradient combined nonvolutionity renders learning susceptible to novel problems. We propose fusing neighboring layers of deeper networks that are trained with random variables.
  arXiv  Detail & Related papers  (2020-01-28T18:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.