Green Multigrid Network
- URL: http://arxiv.org/abs/2407.03593v1
- Date: Thu, 4 Jul 2024 03:02:10 GMT
- Title: Green Multigrid Network
- Authors: Ye Lin, Young Ju Lee, Jiwei Jia,
- Abstract summary: GreenLearning networks (GL) learn Green's function in physical space, making them an interpretable model for capturing unknown solution operators of partial differential equations (PDEs)
We propose a framework named Green Multigrid networks (GreenMGNet), an operator learning algorithm designed for a class of singularityally smooth Green's functions.
Compared with the pioneering GL, the new framework presents itself with better accuracy and efficiency, thereby achieving a significant improvement.
- Score: 6.397295511397678
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: GreenLearning networks (GL) directly learn Green's function in physical space, making them an interpretable model for capturing unknown solution operators of partial differential equations (PDEs). For many PDEs, the corresponding Green's function exhibits asymptotic smoothness. In this paper, we propose a framework named Green Multigrid networks (GreenMGNet), an operator learning algorithm designed for a class of asymptotically smooth Green's functions. Compared with the pioneering GL, the new framework presents itself with better accuracy and efficiency, thereby achieving a significant improvement. GreenMGNet is composed of two technical novelties. First, Green's function is modeled as a piecewise function to take into account its singular behavior in some parts of the hyperplane. Such piecewise function is then approximated by a neural network with augmented output(AugNN) so that it can capture singularity accurately. Second, the asymptotic smoothness property of Green's function is used to leverage the Multi-Level Multi-Integration (MLMI) algorithm for both the training and inference stages. Several test cases of operator learning are presented to demonstrate the accuracy and effectiveness of the proposed method. On average, GreenMGNet achieves $3.8\%$ to $39.15\%$ accuracy improvement. To match the accuracy level of GL, GreenMGNet requires only about $10\%$ of the full grid data, resulting in a $55.9\%$ and $92.5\%$ reduction in training time and GPU memory cost for one-dimensional test problems, and a $37.7\%$ and $62.5\%$ reduction for two-dimensional test problems.
Related papers
- On the Convergence of Multi-objective Optimization under Generalized Smoothness [27.87166415148172]
We study a more general and realistic class of $ell$-smooth loss functions, where $ell$ is a general non-decreasing function norm.
We develop two novel algorithms for $ell$-smooth Generalized Multi-MOO GradientGrad and its variant, Generalized Smooth Multi-MOO descent.
Our algorithms can also guarantee a tighter $mathcalO(epsilon-2)$ in each iteration using more samples.
arXiv Detail & Related papers (2024-05-29T18:36:59Z) - Learning Green's Function Efficiently Using Low-Rank Approximations [44.46178415547532]
A practical limitation of using deep learning for the Green's function is the repeated computationally expensive Monte-Carlo integral approximations.
We propose to learn the Green's function by low-rank decomposition, which results in a novel architecture to remove redundant computations.
arXiv Detail & Related papers (2023-08-01T07:43:46Z) - Cramer Type Distances for Learning Gaussian Mixture Models by Gradient
Descent [0.0]
As of today, few known algorithms can fit or learn Gaussian mixture models.
We propose a distance function called Sliced Cram'er 2-distance for learning general multivariate GMMs.
These features are especially useful for distributional reinforcement learning and Deep Q Networks.
arXiv Detail & Related papers (2023-07-13T13:43:02Z) - On Convergence of Incremental Gradient for Non-Convex Smooth Functions [63.51187646914962]
In machine learning and network optimization, algorithms like shuffle SGD are popular due to minimizing the number of misses and good cache.
This paper delves into the convergence properties SGD algorithms with arbitrary data ordering.
arXiv Detail & Related papers (2023-05-30T17:47:27Z) - BI-GreenNet: Learning Green's functions by boundary integral network [14.008606361378149]
Green's function plays a significant role in both theoretical analysis and numerical computing of partial differential equations.
We develop a new method for computing Green's function with high accuracy.
arXiv Detail & Related papers (2022-04-28T01:42:35Z) - Graph-adaptive Rectified Linear Unit for Graph Neural Networks [64.92221119723048]
Graph Neural Networks (GNNs) have achieved remarkable success by extending traditional convolution to learning on non-Euclidean data.
We propose Graph-adaptive Rectified Linear Unit (GReLU) which is a new parametric activation function incorporating the neighborhood information in a novel and efficient way.
We conduct comprehensive experiments to show that our plug-and-play GReLU method is efficient and effective given different GNN backbones and various downstream tasks.
arXiv Detail & Related papers (2022-02-13T10:54:59Z) - Greenformer: Factorization Toolkit for Efficient Deep Neural Networks [35.47418512373472]
Greenformer is a toolkit to accelerate the computation of neural networks through matrix factorization.
Our experimental results show that Greenformer is effective for a wide range of scenarios.
arXiv Detail & Related papers (2021-09-14T15:27:05Z) - Exploiting Adam-like Optimization Algorithms to Improve the Performance
of Convolutional Neural Networks [82.61182037130405]
gradient descent (SGD) is the main approach for training deep networks.
In this work, we compare Adam based variants based on the difference between the present and the past gradients.
We have tested ensemble of networks and the fusion with ResNet50 trained with gradient descent.
arXiv Detail & Related papers (2021-03-26T18:55:08Z) - SALA: Soft Assignment Local Aggregation for Parameter Efficient 3D
Semantic Segmentation [65.96170587706148]
We focus on designing a point local aggregation function that yields parameter efficient networks for 3D point cloud semantic segmentation.
We explore the idea of using learnable neighbor-to-grid soft assignment in grid-based aggregation functions.
arXiv Detail & Related papers (2020-12-29T20:16:37Z) - Learning outside the Black-Box: The pursuit of interpretable models [78.32475359554395]
This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function.
Our interpretation represents a leap forward from the previous state of the art.
arXiv Detail & Related papers (2020-11-17T12:39:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.