An Imbalanced Learning-based Sampling Method for Physics-informed Neural Networks

• URL: http://arxiv.org/abs/2501.11222v1
• Date: Mon, 20 Jan 2025 02:16:03 GMT
• Title: An Imbalanced Learning-based Sampling Method for Physics-informed Neural Networks
• Authors: Jiaqi Luo, Yahong Yang, Yuan Yuan, Shixin Xu, Wenrui Hao,
• Abstract summary: RSmote is an innovative local adaptive sampling technique tailored to improve the performance of Physics-Informed Neural Networks (PINNs) Traditional residual-based adaptive sampling methods, while effective in enhancing PINN accuracy, often struggle with efficiency and high memory consumption.
• Score: 4.609724496676172
• License:
• Abstract: This paper introduces Residual-based Smote (RSmote), an innovative local adaptive sampling technique tailored to improve the performance of Physics-Informed Neural Networks (PINNs) through imbalanced learning strategies. Traditional residual-based adaptive sampling methods, while effective in enhancing PINN accuracy, often struggle with efficiency and high memory consumption, particularly in high-dimensional problems. RSmote addresses these challenges by targeting regions with high residuals and employing oversampling techniques from imbalanced learning to refine the sampling process. Our approach is underpinned by a rigorous theoretical analysis that supports the effectiveness of RSmote in managing computational resources more efficiently. Through extensive evaluations, we benchmark RSmote against the state-of-the-art Residual-based Adaptive Distribution (RAD) method across a variety of dimensions and differential equations. The results demonstrate that RSmote not only achieves or exceeds the accuracy of RAD but also significantly reduces memory usage, making it particularly advantageous in high-dimensional scenarios. These contributions position RSmote as a robust and resource-efficient solution for solving complex partial differential equations, especially when computational constraints are a critical consideration.

Related papers

• NeuroLifting: Neural Inference on Markov Random Fields at Scale [14.042164101688682]
  Inference in large-scale Markov Random Fields (MRFs) is a critical yet challenging task. This paper introduces NeuroLifting, a novel technique that leverages Graph Neural Networks (GNNs) to re parameterize decision variables in MRFs. By extending traditional lifting techniques into a non-parametric neural network framework, NeuroLifting benefits from the smooth loss landscape of neural networks.
  arXiv  Detail & Related papers  (2024-11-28T06:50:47Z)
• Optimization of Actuarial Neural Networks with Response Surface Methodology [0.0]
  This study utilizes a factorial design and response surface methodology (RSM) to optimize CANN performance. By dropping statistically insignificant hyper parameters, we reduced runs from 288 to 188, with negligible loss in accuracy, achieving near-optimal out-of-sample Poisson deviance loss.
  arXiv  Detail & Related papers  (2024-10-01T15:45:41Z)
• Robust Neural Pruning with Gradient Sampling Optimization for Residual Neural Networks [0.0]
  This research embarks on pioneering the integration of gradient sampling optimization techniques, particularly StochGradAdam, into the pruning process of neural networks. Our main objective is to address the significant challenge of maintaining accuracy in pruned neural models, critical in resource-constrained scenarios.
  arXiv  Detail & Related papers  (2023-12-26T12:19:22Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization [4.0554893636822]
  We introduce a novel approach to deploy large-scale Deep Neural Networks on constrained resources. The method speeds up inference time and aims to reduce memory demand and power consumption.
  arXiv  Detail & Related papers  (2022-12-25T15:40:05Z)
• FasterPose: A Faster Simple Baseline for Human Pose Estimation [65.8413964785972]
  We propose a design paradigm for cost-effective network with LR representation for efficient pose estimation, named FasterPose. We study the training behavior of FasterPose, and formulate a novel regressive cross-entropy (RCE) loss function for accelerating the convergence. Compared with the previously dominant network of pose estimation, our method reduces 58% of the FLOPs and simultaneously gains 1.3% improvement of accuracy.
  arXiv  Detail & Related papers  (2021-07-07T13:39:08Z)
• Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization [85.84019017587477]
  Distributionally robust supervised learning is emerging as a key paradigm for building reliable machine learning systems for real-world applications. Existing algorithms for solving Wasserstein DRSL involve solving complex subproblems or fail to make use of gradients. We revisit Wasserstein DRSL through the lens of min-max optimization and derive scalable and efficiently implementable extra-gradient algorithms.
  arXiv  Detail & Related papers  (2021-04-27T16:56:09Z)
• Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks [82.33619654835348]
  Intelligent surface (IRS) has been employed to reshape the wireless channels by controlling individual scattering elements' phase shifts. Due to the large size of scattering elements, the passive beamforming is typically challenged by the high computational complexity. In this article, we focus on machine learning (ML) approaches for performance in IRS-assisted wireless networks.
  arXiv  Detail & Related papers  (2020-08-29T08:39:43Z)
• ALF: Autoencoder-based Low-rank Filter-sharing for Efficient Convolutional Neural Networks [63.91384986073851]
  We propose the autoencoder-based low-rank filter-sharing technique technique (ALF) ALF shows a reduction of 70% in network parameters, 61% in operations and 41% in execution time, with minimal loss in accuracy.
  arXiv  Detail & Related papers  (2020-07-27T09:01:22Z)
• Deep Adaptive Inference Networks for Single Image Super-Resolution [72.7304455761067]
  Single image super-resolution (SISR) has witnessed tremendous progress in recent years owing to the deployment of deep convolutional neural networks (CNNs) In this paper, we take a step forward to address this issue by leveraging the adaptive inference networks for deep SISR (AdaDSR) Our AdaDSR involves an SISR model as backbone and a lightweight adapter module which takes image features and resource constraint as input and predicts a map of local network depth.
  arXiv  Detail & Related papers  (2020-04-08T10:08:20Z)
• Distributionally Robust Deep Learning using Hardness Weighted Sampling [5.277562268045534]
  We propose a principled and efficient algorithm for DRO in machine learning that is particularly suited in the context of deep learning. Our experiments on fetal brain 3D MRI segmentation and brain tumor segmentation in MRI demonstrate the feasibility and the usefulness of our approach.
  arXiv  Detail & Related papers  (2020-01-08T18:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.