Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2502.12384v1
• Date: Mon, 17 Feb 2025 23:45:23 GMT
• Title: Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks
• Authors: Yequan Zhao, Xinling Yu, Xian Xiao, Zhixiong Chen, Ziyue Liu, Geza Kurczveil, Raymond G. Beausoleil, Sijia Liu, Zheng Zhang,
• Abstract summary: Physics-informed neural networks (PINNs) have shown promise in solving partial differential equations (PDEs) Photonic computing offers a potential solution to achieve this goal because of its ultra-high operation speed. This paper proposes a completely back-propagation-free (BP-free) and highly salable framework for training real-size PINNs on silicon photonic platforms.
• Score: 12.726911225088443
• License:
• Abstract: Physics-informed neural networks (PINNs) have shown promise in solving partial differential equations (PDEs), with growing interest in their energy-efficient, real-time training on edge devices. Photonic computing offers a potential solution to achieve this goal because of its ultra-high operation speed. However, the lack of photonic memory and the large device sizes prevent training real-size PINNs on photonic chips. This paper proposes a completely back-propagation-free (BP-free) and highly salable framework for training real-size PINNs on silicon photonic platforms. Our approach involves three key innovations: (1) a sparse-grid Stein derivative estimator to avoid the BP in the loss evaluation of a PINN, (2) a dimension-reduced zeroth-order optimization via tensor-train decomposition to achieve better scalability and convergence in BP-free training, and (3) a scalable on-chip photonic PINN training accelerator design using photonic tensor cores. We validate our numerical methods on both low- and high-dimensional PDE benchmarks. Through circuit simulation based on real device parameters, we further demonstrate the significant performance benefit (e.g., real-time training, huge chip area reduction) of our photonic accelerator.

Related papers

• Asymmetrical estimator for training encapsulated deep photonic neural networks [10.709758849326061]
  Photonic neural networks (PNNs) are fast in-propagation and high bandwidth paradigms. We introduce the asymmetrical training (AsyT) method, tailored for encapsulated DPNNs. AsyT offers a lightweight solution for DPNNs with minimum readouts, fast and energy-efficient operation, and minimum system footprint.
  arXiv  Detail & Related papers  (2024-05-28T17:27:20Z)
• Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free Optical PINN Training [5.809283001227614]
  This paper develops an on-chip training framework for physics-informed neural networks (PINNs) It aims to solve high-dimensional PDEs with fJ/MAC photonic power consumption and ultra-low latency. This is the first real-size optical PINN training framework that can be applied to solve high-dimensional PDEs.
  arXiv  Detail & Related papers  (2023-12-31T07:10:15Z)
• EPIM: Efficient Processing-In-Memory Accelerators based on Epitome [78.79382890789607]
  We introduce the Epitome, a lightweight neural operator offering convolution-like functionality. On the software side, we evaluate epitomes' latency and energy on PIM accelerators. We introduce a PIM-aware layer-wise design method to enhance their hardware efficiency.
  arXiv  Detail & Related papers  (2023-11-12T17:56:39Z)
• Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks [15.188785164091987]
  Backward propagation (BP) is widely used to compute the gradients in neural network training. It is hard to implement BP on edge devices due to the lack of hardware and software resources to support automatic differentiation. This paper presents a completely BP-free framework that only requires forward propagation to train realistic neural networks.
  arXiv  Detail & Related papers  (2023-08-18T23:56:50Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• TT-PINN: A Tensor-Compressed Neural PDE Solver for Edge Computing [7.429526302331948]
  Physics-informed neural networks (PINNs) have been increasingly employed due to their capability of modeling complex physics systems. This paper proposes an end-to-end compressed PINN based on Helmholtz-Train decomposition.
  arXiv  Detail & Related papers  (2022-07-04T23:56:27Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method [70.19159220248805]
  Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs) We propose the generative adversarial neural network (GA-PINN), which integrates the generative adversarial (GA) mechanism with the structure of PINNs. Inspired from the weighting strategy of the Adaboost method, we then introduce a point-weighting (PW) method to improve the training efficiency of PINNs.
  arXiv  Detail & Related papers  (2022-05-18T06:50:44Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization [10.005026783940682]
  Silicon-photonics-based optical neural network (ONN) is a promising hardware platform that could represent a paradigm shift in efficient AI. In this work, we propose a closed-loop ONN on-chip learning framework L2ight to enable scalable ONN mapping and efficient in-situ learning.
  arXiv  Detail & Related papers  (2021-10-27T22:53:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.