Frequency-adaptive Multi-scale Deep Neural Networks

• URL: http://arxiv.org/abs/2410.00053v1
• Date: Sat, 28 Sep 2024 14:49:23 GMT
• Title: Frequency-adaptive Multi-scale Deep Neural Networks
• Authors: Jizu Huang, Rukang You, Tao Zhou,
• Abstract summary: We propose frequency-adaptive MscaleDNNs for approximating high frequency functions. These MscaleDNNs improve accuracy by two to three orders of magnitude compared to standard MscaleDNNs.
• Score: 6.338572283139395
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-scale deep neural networks (MscaleDNNs) with downing-scaling mapping have demonstrated superiority over traditional DNNs in approximating target functions characterized by high frequency features. However, the performance of MscaleDNNs heavily depends on the parameters in the downing-scaling mapping, which limits their broader application. In this work, we establish a fitting error bound to explain why MscaleDNNs are advantageous for approximating high frequency functions. Building on this insight, we construct a hybrid feature embedding to enhance the accuracy and robustness of the downing-scaling mapping. To reduce the dependency of MscaleDNNs on parameters in the downing-scaling mapping, we propose frequency-adaptive MscaleDNNs, which adaptively adjust these parameters based on a posterior error estimate that captures the frequency information of the fitted functions. Numerical examples, including wave propagation and the propagation of a localized solution of the schr$\ddot{\text{o}}$dinger equation with a smooth potential near the semi-classical limit, are presented. These examples demonstrate that the frequency-adaptive MscaleDNNs improve accuracy by two to three orders of magnitude compared to standard MscaleDNNs.

Related papers

• Neuromorphic Wireless Split Computing with Multi-Level Spikes [69.73249913506042]
  In neuromorphic computing, spiking neural networks (SNNs) perform inference tasks, offering significant efficiency gains for workloads involving sequential data. Recent advances in hardware and software have demonstrated that embedding a few bits of payload in each spike exchanged between the spiking neurons can further enhance inference accuracy. This paper investigates a wireless neuromorphic split computing architecture employing multi-level SNNs.
  arXiv  Detail & Related papers  (2024-11-07T14:08:35Z)
• Data-driven localized waves and parameter discovery in the massive Thirring model via extended physics-informed neural networks with interface zones [3.522950356329991]
  We study data-driven localized wave solutions and parameter discovery in the massive Thirring (MT) model via the deep learning. For higher-order localized wave solutions, we employ the extended PINNs (XPINNs) with domain decomposition. Experimental results show that this improved version of XPINNs reduce the complexity of computation with faster convergence rate.
  arXiv  Detail & Related papers  (2023-09-29T13:50:32Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• PINNup: Robust neural network wavefield solutions using frequency upscaling and neuron splitting [0.0]
  We propose a novel implementation of PINN using frequency upscaling and neuron splitting. The proposed PINN exhibits notable superiority in terms of convergence and accuracy. It can achieve neuron based high-frequency wavefield solutions with a two-hidden-layer model.
  arXiv  Detail & Related papers  (2021-09-29T16:35:50Z)
• Neural Calibration for Scalable Beamforming in FDD Massive MIMO with Implicit Channel Estimation [10.775558382613077]
  Channel estimation and beamforming play critical roles in frequency-division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems. We propose a deep learning-based approach that directly optimize the beamformers at the base station according to the received uplink pilots. A neural calibration method is proposed to improve the scalability of the end-to-end design.
  arXiv  Detail & Related papers  (2021-08-03T14:26:14Z)
• Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains [12.09637784919702]
  We propose multi-scale deep neural networks (MscaleDNNs) using the idea of radial scaling in frequency domain and activation functions with compact support. As a result, the MscaleDNNs achieve fast uniform convergence over multiple scales.
  arXiv  Detail & Related papers  (2020-07-22T05:28:03Z)
• Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks [107.77595511218429]
  In this paper, we investigate the empirical Rademacher complexity related to intermediate layers of deep neural networks. We propose a feature distortion method (Disout) for addressing the aforementioned problem. The superiority of the proposed feature map distortion for producing deep neural network with higher testing performance is analyzed and demonstrated.
  arXiv  Detail & Related papers  (2020-02-23T13:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.