Green Multigrid Network

• URL: http://arxiv.org/abs/2407.03593v1
• Date: Thu, 4 Jul 2024 03:02:10 GMT
• Title: Green Multigrid Network
• Authors: Ye Lin, Young Ju Lee, Jiwei Jia,
• Abstract summary: GreenLearning networks (GL) learn Green's function in physical space, making them an interpretable model for capturing unknown solution operators of partial differential equations (PDEs) We propose a framework named Green Multigrid networks (GreenMGNet), an operator learning algorithm designed for a class of singularityally smooth Green's functions. Compared with the pioneering GL, the new framework presents itself with better accuracy and efficiency, thereby achieving a significant improvement.
• Score: 6.397295511397678
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: GreenLearning networks (GL) directly learn Green's function in physical space, making them an interpretable model for capturing unknown solution operators of partial differential equations (PDEs). For many PDEs, the corresponding Green's function exhibits asymptotic smoothness. In this paper, we propose a framework named Green Multigrid networks (GreenMGNet), an operator learning algorithm designed for a class of asymptotically smooth Green's functions. Compared with the pioneering GL, the new framework presents itself with better accuracy and efficiency, thereby achieving a significant improvement. GreenMGNet is composed of two technical novelties. First, Green's function is modeled as a piecewise function to take into account its singular behavior in some parts of the hyperplane. Such piecewise function is then approximated by a neural network with augmented output(AugNN) so that it can capture singularity accurately. Second, the asymptotic smoothness property of Green's function is used to leverage the Multi-Level Multi-Integration (MLMI) algorithm for both the training and inference stages. Several test cases of operator learning are presented to demonstrate the accuracy and effectiveness of the proposed method. On average, GreenMGNet achieves $3.8\%$ to $39.15\%$ accuracy improvement. To match the accuracy level of GL, GreenMGNet requires only about $10\%$ of the full grid data, resulting in a $55.9\%$ and $92.5\%$ reduction in training time and GPU memory cost for one-dimensional test problems, and a $37.7\%$ and $62.5\%$ reduction for two-dimensional test problems.

Related papers

• Fast Semi-supervised Learning on Large Graphs: An Improved Green-function Method [93.04936336359502]
  In graph-based semi-supervised learning, the Green-function method is a classical method that works by computing the Green's function in the graph space. We make a detailed analysis on it and propose a novel method from the perspective of optimization. Unlike the original method, our improved method can also apply two accelerating techniques, Gaussian Elimination, and Anchored Graphs.
  arXiv  Detail & Related papers  (2024-11-04T04:27:18Z)
• The Optimization Landscape of SGD Across the Feature Learning Strength [102.1353410293931]
  We study the effect of scaling $gamma$ across a variety of models and datasets in the online training setting. We find that optimal online performance is often found at large $gamma$. Our findings indicate that analytical study of the large-$gamma$ limit may yield useful insights into the dynamics of representation learning in performant models.
  arXiv  Detail & Related papers  (2024-10-06T22:30:14Z)
• Learning Green's Function Efficiently Using Low-Rank Approximations [44.46178415547532]
  A practical limitation of using deep learning for the Green's function is the repeated computationally expensive Monte-Carlo integral approximations. We propose to learn the Green's function by low-rank decomposition, which results in a novel architecture to remove redundant computations.
  arXiv  Detail & Related papers  (2023-08-01T07:43:46Z)
• Cramer Type Distances for Learning Gaussian Mixture Models by Gradient Descent [0.0]
  As of today, few known algorithms can fit or learn Gaussian mixture models. We propose a distance function called Sliced Cram'er 2-distance for learning general multivariate GMMs. These features are especially useful for distributional reinforcement learning and Deep Q Networks.
  arXiv  Detail & Related papers  (2023-07-13T13:43:02Z)
• DOGE-Train: Discrete Optimization on GPU with End-to-end Training [28.795080637690095]
  We present a fast, scalable, data-driven approach for solving relaxations of 0-1 integer linear programs. We use a combination of graph neural networks (GNN) and the Lagrange decomposition based algorithm FastDOG.
  arXiv  Detail & Related papers  (2022-05-23T21:09:41Z)
• BI-GreenNet: Learning Green's functions by boundary integral network [14.008606361378149]
  Green's function plays a significant role in both theoretical analysis and numerical computing of partial differential equations. We develop a new method for computing Green's function with high accuracy.
  arXiv  Detail & Related papers  (2022-04-28T01:42:35Z)
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks [64.92221119723048]
  Graph Neural Networks (GNNs) have achieved remarkable success by extending traditional convolution to learning on non-Euclidean data. We propose Graph-adaptive Rectified Linear Unit (GReLU) which is a new parametric activation function incorporating the neighborhood information in a novel and efficient way. We conduct comprehensive experiments to show that our plug-and-play GReLU method is efficient and effective given different GNN backbones and various downstream tasks.
  arXiv  Detail & Related papers  (2022-02-13T10:54:59Z)
• Greenformer: Factorization Toolkit for Efficient Deep Neural Networks [35.47418512373472]
  Greenformer is a toolkit to accelerate the computation of neural networks through matrix factorization. Our experimental results show that Greenformer is effective for a wide range of scenarios.
  arXiv  Detail & Related papers  (2021-09-14T15:27:05Z)
• Exploiting Adam-like Optimization Algorithms to Improve the Performance of Convolutional Neural Networks [82.61182037130405]
  gradient descent (SGD) is the main approach for training deep networks. In this work, we compare Adam based variants based on the difference between the present and the past gradients. We have tested ensemble of networks and the fusion with ResNet50 trained with gradient descent.
  arXiv  Detail & Related papers  (2021-03-26T18:55:08Z)
• SALA: Soft Assignment Local Aggregation for Parameter Efficient 3D Semantic Segmentation [65.96170587706148]
  We focus on designing a point local aggregation function that yields parameter efficient networks for 3D point cloud semantic segmentation. We explore the idea of using learnable neighbor-to-grid soft assignment in grid-based aggregation functions.
  arXiv  Detail & Related papers  (2020-12-29T20:16:37Z)
• Learning outside the Black-Box: The pursuit of interpretable models [78.32475359554395]
  This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function. Our interpretation represents a leap forward from the previous state of the art.
  arXiv  Detail & Related papers  (2020-11-17T12:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.