Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2605.10136v1
• Date: Mon, 11 May 2026 07:46:07 GMT
• Title: Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks
• Abstract summary: We show that PINN gradient conflict is not a uniform failure mode with one universal remedy.<n>We identify distinct PINN gradient-conflict regimes, each associated with a different intervention class.
• Score: 3.1861308132183375
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physics-informed neural networks (PINNs) train a single neural approximation by minimizing multiple physics- and data-derived losses, but the gradients of these losses often interfere and can stall optimization. Existing remedies typically treat this pathology either through scalar loss balancing or full-parameter-space gradient surgery, leaving it unclear which intervention is most appropriate. We show that PINN gradient conflict is not a uniform failure mode with one universal remedy. Instead, we identify distinct PINN gradient-conflict regimes, each associated with a different intervention class. Persistent directional conflict may require separate loss-indexed parameter subspaces, magnitude imbalance often favors scalar reweighting, and low or transient conflict may require no extra mitigation. To select between scalar reweighting and a lightweight architectural intervention, we propose a diagnostic-first framework. It profiles a 1000-step unmodified PINN run and, when intervention is warranted, uses one low-rank adapter per loss to create explicit loss-indexed parameter subspaces attached to a shared PINN trunk, providing each loss with a direct gradient pathway. Across more than 60 PDE configurations, including forward, inverse, multi-physics, parameter-varying, and high-dimensional problems up to 50D, persistent directional conflict dominates standard forward $K=3$ benchmarks and a natural $K=4$ thermoelastic system, where adapters combined with reweighting yield significant improvements. In contrast, $K=3$ inverse problems and natural $K=5$ and $K=6$ multi-physics systems are largely magnitude-dominated and often favor reweighting alone, while full-parameter-space gradient surgery can fail on heterogeneous parameter spaces.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• Adjoint Method versus Physics-Informed Neural Networks in PDE-Constrained Inverse Problems [48.470700002941136]
  Inverse problems governed by partial differential equations (PDEs) are central to computational mechanics and are commonly solved by adjoint-based optimization.<n> physics-informed neural networks (PINNs) have emerged as a flexible alternative.<n>We present a fair comparison of adjoint optimization and PINNs for PDE-constrained inverse problems.
  arXiv  Detail & Related papers  (2026-06-10T17:07:09Z)
• Loss Landscape Diagnosis for Gradient-Based Gray-Scott System Inversion: Disentangling the Roles of PINN Components [6.626016846098371]
  Theory of gradient-based inversion of reaction-diffusion systems is typically approached via surrogate models or physics-informed neural networks (PINNs)<n>We pursue this direct route as a diagnostic probe, backpropagating a steady-state loss through unrolled Gray-Scott simulation to recover its parameters, with no surrogate or neural-network augmentation.<n>Reading this minimal setup as an ablation of PINN, we disentangle each component's role.<n>These findings carry concrete design implications for PINN-type methods and a broader on when added dimensions actually help.
  arXiv  Detail & Related papers  (2026-06-09T01:13:00Z)
• Loss-Conditional PINNs for Parametric PDE Families [45.945826972787415]
  Physics-informed neural networks (PINNs) approximate solutions of ODEs and PDEs by minimising a weighted combination of residual, boundary, initial, and data losses.<n>We introduce LC-PINN, which adapts the lossconditional training of Dosovitskiy and Djolonga ( 2020) to the PDE-residual setting.<n>It stays fully physics-informed but amortises training over a parametric family.
  arXiv  Detail & Related papers  (2026-06-03T04:02:10Z)
• Mitigating Gradient Pathology in PINNs through Aligned Constraint [14.405886383974218]
  PINNs are powerful for solving Partial Differential Equations (PDEs)<n>Their training is often paralyzed by gradient pathology.<n>We propose Constraint-Aligned loss lifting to mitigate gradient conflicts.
  arXiv  Detail & Related papers  (2026-05-24T10:58:12Z)
• Supervised Metric Regularization Through Alternating Optimization for Multi-Regime Physics-Informed Neural Networks [0.0]
  PINNs often face challenges when modeling dynamical systems with sharp regime transitions, such as bifurcations.<n>We propose a Topology-Aware PINN (TAPINN) that aims to mitigate this challenge by the latent space via Supervised Metric Regularization.<n>Preliminary experiments on the Duffing demonstrate that while standard baselines suffer from spectral bias and high-capacity gradient networks overfit, our approach achieves stable convergence with 2.18x lower variance than a multi-output Sobolev Error baseline, and 5x fewer parameters than a hypernetwork-based alternative.
  arXiv  Detail & Related papers  (2026-02-10T17:06:57Z)
• APRIL: Auxiliary Physically-Redundant Information in Loss - A physics-informed framework for parameter estimation with a gravitational-wave case study [0.0]
  Physics-Informed Neural Networks (PINNs) embed the partial differential equations governing the system under study directly into the training of Neural Networks.<n>We present a complementary approach by including auxiliary physically-redundant information in loss.<n>We mathematically demonstrate that these terms preserve the true physical minimum while reshaping the loss landscape.
  arXiv  Detail & Related papers  (2025-10-15T15:34:19Z)
• Neural PDE Solvers with Physics Constraints: A Comparative Study of PINNs, DRM, and WANs [1.131316248570352]
  Partial equations (PDEs) underpin models across science and engineering, yet analytical solutions are atypical and classical mesh-based solvers can be costly in high dimensions.<n>This dissertation presents a unified comparison of three mesh-free neural PDE solvers, physics-informed neural networks (PINNs), the deep Ritz method (DRM), and weak adversarial networks (WANs), on Poisson problems (up to 5D) and the time-independent Schr"odinger equation in 1D/2D.
  arXiv  Detail & Related papers  (2025-10-09T13:41:51Z)
• Targeted Deep Architectures: A TMLE-Based Framework for Robust Causal Inference in Neural Networks [41.07059178065932]
  We propose Targeted Deep Architectures (TDA), a new framework that embeds Targeted Likelihood Estimation (TMLE) directly into the network's parameter space.<n>TDA partitions model parameters and iteratively updates them along a targeting gradient.<n>This procedure yields plug-in estimates that remove first-order bias and produce valid confidence intervals.
  arXiv  Detail & Related papers  (2025-07-16T17:24:06Z)
• The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions [51.68215326304272]
  We show that even small perturbations reliably cause otherwise identical training trajectories to diverge-an effect that diminishes rapidly over training time.<n>Our findings provide insights into neural network training stability, with practical implications for fine-tuning, model merging, and diversity of model ensembles.
  arXiv  Detail & Related papers  (2025-06-16T08:35:16Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Enabling Automatic Differentiation with Mollified Graph Neural Operators [73.52999622724101]
  We propose the mollified graph neural operator ($m$GNO), the first method to leverage automatic differentiation and compute exact gradients on arbitrary geometries.<n>For a PDE example on regular grids, $m$GNO paired with autograd reduced the L2 relative data error by 20x compared to finite differences.<n>It can also solve PDEs on unstructured point clouds seamlessly, using physics losses only, at resolutions vastly lower than those needed for finite differences to be accurate enough.
  arXiv  Detail & Related papers  (2025-04-11T06:16:30Z)
• Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs [93.82811501035569]
  We introduce a new data efficient and highly parallelizable operator learning approach with reduced memory requirement and better generalization. MG-TFNO scales to large resolutions by leveraging local and global structures of full-scale, real-world phenomena. We demonstrate superior performance on the turbulent Navier-Stokes equations where we achieve less than half the error with over 150x compression.
  arXiv  Detail & Related papers  (2023-09-29T20:18:52Z)
• Adaptive Self-supervision Algorithms for Physics-informed Neural Networks [59.822151945132525]
  Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function. We study the impact of the location of the collocation points on the trainability of these models. We propose a novel adaptive collocation scheme which progressively allocates more collocation points to areas where the model is making higher errors.
  arXiv  Detail & Related papers  (2022-07-08T18:17:06Z)
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data [47.49194807524502]
  We propose LordNet, a tunable and efficient neural network for modeling entanglements. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements can be well modeled by the LordNet.
  arXiv  Detail & Related papers  (2022-06-19T14:41:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.