Physics-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning

• URL: http://arxiv.org/abs/2605.05217v1
• Date: Fri, 17 Apr 2026 01:48:54 GMT
• Title: Physics-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning
• Abstract summary: We propose a self-supervised physics-informed neural network (PINN) framework that adaptively balances physics-based and data-driven supervision.<n>Our framework provides a general recipe for embedding physics adaptively into neural networks, offering a robust and reproducible approach for data-scarce problems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a self-supervised physics-informed neural network (PINN) framework that adaptively balances physics-based and data-driven supervision for scientific machine learning under data scarcity. Unlike prior PINNs that rely on fixed or heuristic weighting of physics residuals and data loss, our approach introduces a learnable blending neuron that dynamically adjusts the relative contribution of each term based on their uncertainties. This mechanism enables stable training and improved generalization without manual tuning. To further enhance efficiency, we integrate a transfer learning strategy that reuses representations from related domains and adapts them to new physical systems with limited data. We validate the framework for the prediction of heat transfer in liquid-metal miniature heat sinks using only 87 CFD datapoints, where the adaptive PINN achieves an error <8%, outperforming shallow neural networks, kernel methods, and physics-only baselines. Our framework provides a general recipe for embedding physics adaptively into neural networks, offering a robust and reproducible approach for data-scarce problems across various scientific domains, including fluid dynamics and material modeling.

Related papers

• Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks [56.42974363744522]
  We study the learning dynamics of random feedback local online (RFLO) and truncated backpropagation through time (tBPTT)<n>Our work provides analytical insight into how locality constraints shape learning dynamics, with implications for neuroscientific models of learning and alternative optimization approaches for RNNs.
  arXiv  Detail & Related papers  (2026-05-29T18:19:45Z)
• Engineering Hybrid Physics-Informed Neural Networks for Next-Generation Electricity Systems: A State-of-the-Art Review [0.0]
  Physics-informed machine learning (PIML) addresses limitations by embedding governing equations directly into the learning process.<n>PIML enables a paradigm shift from black-box data-driven methods to transparent, physics-informed strategies.
  arXiv  Detail & Related papers  (2026-05-21T02:25:05Z)
• Kirchhoff-Inspired Neural Networks for Evolving High-Order Perception [51.28469959548178]
  We propose the Kirchhoff-Inspired Neural Network (KINN), a state-variable-based network architecture constructed based on Kirchhoff's current law.<n>KINN derives numerically stable state updates from fundamental ordinary differential equations, enabling the explicit decoupling and encoding of higher-order evolutionary components.
  arXiv  Detail & Related papers  (2026-03-25T06:17:31Z)
• Limitations of Physics-Informed Neural Networks: a Study on Smart Grid Surrogation [29.49941497527361]
  PINNs present a transformative approach for smart grid modeling by integrating physical laws directly into learning frameworks.<n>This paper evaluates PINNs' capabilities as surrogate models for smart grid dynamics.<n>We demonstrate PINNs' superior generalization, outperforming data-driven models in error reduction.
  arXiv  Detail & Related papers  (2025-08-29T12:15:32Z)
• Optimal Linear Baseline Models for Scientific Machine Learning [37.69303106863453]
  We develop a unified theoretical framework for analyzing linear encoder-decoder architectures.<n>We derive closed-form, rank-constrained linear optimal mappings for forward modeling and inverse recovery tasks.<n>This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.
  arXiv  Detail & Related papers  (2025-08-07T20:17:07Z)
• Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions [44.44497277876625]
  Physics-Informed Neural Networks (PINNs) are deep learning models that incorporate the governing physical laws of a system into the learning process.<n>We introduce a transfer learning (TL) method to improve the extrapolation capability of PINNs.<n>We demonstrate that our method achieves an average of 40% reduction in relative L2 error and an average of 50% reduction in mean absolute error.
  arXiv  Detail & Related papers  (2025-07-16T22:19:53Z)
• Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks [45.752780408098765]
  Physics-informed neural networks (PINNs) offer a powerful approach to solving partial differential equations (PDEs)<n>PINNs have recently emerged as a valuable tool in the field of scientific machine learning.
  arXiv  Detail & Related papers  (2025-06-19T16:21:14Z)
• Physics Informed Constrained Learning of Dynamics from Static Data [8.346864633675414]
  A physics-informed neural network (PINN) models the dynamics of a system by integrating the governing physical laws into the architecture of a neural network.<n>Existing PINN frameworks rely on fully observed time-course data, the acquisition of which could be prohibitive for many systems.<n>In this study, we developed a new PINN learning paradigm, namely Constrained Learning, that enables the approximation of first-order derivatives or motions using non-time course or partially observed data.
  arXiv  Detail & Related papers  (2025-04-17T06:06:53Z)
• Adapting Physics-Informed Neural Networks to Improve ODE Optimization in Mosquito Population Dynamics [0.019972837513980313]
  We propose a PINN framework with several improvements for forward and inverse problems for ODE systems.<n>The framework tackles the gradient imbalance and stiff problems posed by mosquito ordinary differential equations.<n>Preliminary results indicate that physics-informed machine learning holds significant potential for advancing the study of ecological systems.
  arXiv  Detail & Related papers  (2024-06-07T17:40:38Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• Physics-Informed Neural Networks with Hard Linear Equality Constraints [9.101849365688905]
  This work proposes a novel physics-informed neural network, KKT-hPINN, which rigorously guarantees hard linear equality constraints. Experiments on Aspen models of a stirred-tank reactor unit, an extractive distillation subsystem, and a chemical plant demonstrate that this model can further enhance the prediction accuracy.
  arXiv  Detail & Related papers  (2024-02-11T17:40:26Z)
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation [49.44309457870649]
  Layer-wise Feedback feedback (LFP) is a novel training principle for neural network-like predictors.<n>LFP decomposes a reward to individual neurons based on their respective contributions.<n>Our method then implements a greedy reinforcing approach helpful parts of the network and weakening harmful ones.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations [44.89798007370551]
  This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions.
  arXiv  Detail & Related papers  (2022-03-02T19:09:52Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Physics informed neural networks for continuum micromechanics [68.8204255655161]
  Recently, physics informed neural networks have successfully been applied to a broad variety of problems in applied mathematics and engineering. Due to the global approximation, physics informed neural networks have difficulties in displaying localized effects and strong non-linear solutions by optimization. It is shown, that the domain decomposition approach is able to accurately resolve nonlinear stress, displacement and energy fields in heterogeneous microstructures obtained from real-world $mu$CT-scans.
  arXiv  Detail & Related papers  (2021-10-14T14:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.