PDE-based Group Equivariant Convolutional Neural Networks

• URL: http://arxiv.org/abs/2001.09046v6
• Date: Mon, 30 May 2022 19:05:29 GMT
• Title: PDE-based Group Equivariant Convolutional Neural Networks
• Authors: Bart Smets, Jim Portegies, Erik Bekkers, Remco Duits
• Abstract summary: We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs) In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. We present experiments to demonstrate the strength of the proposed PDE-G-CNNs in increasing the performance of deep learning based imaging applications.
• Score: 1.949912057689623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to the standard translation equivariance of CNNs. Having all the desired symmetries included in the design obviates the need to include them by means of costly techniques such as data augmentation. We will discuss our PDE-based G-CNNs (PDE-G-CNNs) in a general homogeneous space setting while also going into the specifics of our primary case of interest: roto-translation equivariance. We solve the PDE of interest by a combination of linear group convolutions and non-linear morphological group convolutions with analytic kernel approximations that we underpin with formal theorems. Our kernel approximations allow for fast GPU-implementation of the PDE-solvers, we release our implementation with this article in the form of the LieTorch extension to PyTorch, available at https://gitlab.com/bsmetsjr/lietorch . Just like for linear convolution a morphological convolution is specified by a kernel that we train in our PDE-G-CNNs. In PDE-G-CNNs we do not use non-linearities such as max/min-pooling and ReLUs as they are already subsumed by morphological convolutions. We present a set of experiments to demonstrate the strength of the proposed PDE-G-CNNs in increasing the performance of deep learning based imaging applications with far fewer parameters than traditional CNNs.

Related papers

• RoPINN: Region Optimized Physics-Informed Neural Networks [66.38369833561039]
  Physics-informed neural networks (PINNs) have been widely applied to solve partial differential equations (PDEs) This paper proposes and theoretically studies a new training paradigm as region optimization. A practical training algorithm, Region Optimized PINN (RoPINN), is seamlessly derived from this new paradigm.
  arXiv  Detail & Related papers  (2024-05-23T09:45:57Z)
• PDE-CNNs: Axiomatic Derivations and Applications [0.1874930567916036]
  Group Convolutional Neural Networks (PDE-G-CNNs) utilize solvers of geometrically meaningful evolution PDEs as substitutes for the conventional components in G-CNNs. We experimentally confirm for small networks that PDE-CNNs offer fewer parameters, increased performance, and better data efficiency when compared to CNNs.
  arXiv  Detail & Related papers  (2024-03-22T13:11:26Z)
• Geometric Generative Models based on Morphological Equivariant PDEs and GANs [3.6498648388765513]
  We propose a geometric generative model based on an equivariant partial differential equation (PDE) for group convolution neural networks (G-CNNs) The proposed geometric morphological GAN (GM-GAN) is obtained by using the proposed morphological equivariant convolutions in PDE-G-CNNs. Preliminary results show that GM-GAN model outperforms classical GAN.
  arXiv  Detail & Related papers  (2024-03-22T01:02:09Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Lie Point Symmetry and Physics Informed Networks [59.56218517113066]
  We propose a loss function that informs the network about Lie point symmetries in the same way that PINN models try to enforce the underlying PDE through a loss function. Our symmetry loss ensures that the infinitesimal generators of the Lie group conserve the PDE solutions. Empirical evaluations indicate that the inductive bias introduced by the Lie point symmetries of the PDEs greatly boosts the sample efficiency of PINNs.
  arXiv  Detail & Related papers  (2023-11-07T19:07:16Z)
• MAgNet: Mesh Agnostic Neural PDE Solver [68.8204255655161]
  Climate predictions require fine-temporal resolutions to resolve all turbulent scales in the fluid simulations. Current numerical model solveers PDEs on grids that are too coarse (3km to 200km on each side) We design a novel architecture that predicts the spatially continuous solution of a PDE given a spatial position query.
  arXiv  Detail & Related papers  (2022-10-11T14:52:20Z)
• Analysis of (sub-)Riemannian PDE-G-CNNs [1.9249287163937971]
  Group equivariant convolutional neural networks (G-CNNs) have been successfully applied in geometric deep learning. We show that the previously suggested approximative morphological kernels do not always accurately approximate the exact kernels. We provide new theorems with better error estimates of the approximative kernels, and prove that they all carry the same reflectional symmetries as the exact ones.
  arXiv  Detail & Related papers  (2022-10-03T13:47:47Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence [0.0]
  Recent research has used deep learning to develop partial differential equation (PDE) models in science and engineering. We rigorously study the optimization of a class of linear elliptic PDEs with neural network terms. We train a neural network model for an application in fluid mechanics, in which the neural network functions as a closure model for the Reynolds-averaged Navier-Stokes equations.
  arXiv  Detail & Related papers  (2021-05-18T16:04:33Z)
• dNNsolve: an efficient NN-based PDE solver [62.997667081978825]
  We introduce dNNsolve, that makes use of dual Neural Networks to solve ODEs/PDEs. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions.
  arXiv  Detail & Related papers  (2021-03-15T19:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.