Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients

• URL: http://arxiv.org/abs/2609.14841v1
• Date: Sun, 13 Sep 2026 23:22:05 GMT
• Title: Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients
• Abstract summary: We bridge domain decomposition with projection-based gradient surgery to mitigate conflicting gradients that degrade training.<n>We show that Norm-PCGrad consistently achieves the lowest relative $L$ error compared to training without gradient surgery.<n>We additionally demonstrate that gradient surgery extends to physics-informed Kolmogorov-Arnold Networks (KANPIs) yielding substantial accuracy improvements for 3D domain decomposition.
• Score: 0.30586855806896046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific machine learning methods such as physics-informed neural networks (PINNs) increasingly rely on domain decomposition for better scalability while solving partial differential equations (PDEs) over complex geometries, yet the resulting composite loss comprising residual, boundary, and interface terms is highly susceptible to conflicting gradients that degrade training. This work bridges domain decomposition with projection-based gradient surgery to systematically mitigate such conflicts in 2D and 3D settings. We evaluate two existing projection-based algorithms, PCGrad and ConFIG, and identify their performance degradation in specific scenarios such as 3D domains with multiple overlapping interfaces. To address this limitation, we propose Norm-PCGrad, a normalized variant that achieves state-of-the-art accuracy across a range of 2D and 3D domain decomposition problems. Across the benchmarks considered, Norm-PCGrad consistently achieves the lowest relative $L_2$ error compared to training without gradient surgery as well as to existing algorithms such as PCGrad and ConFIG, while incurring negligible additional computational overhead. To improve computational efficiency of domain decomposition frameworks such as Extended PINN (XPINN), we propose replacing vanilla PINNs in selected subdomains with separable architectures such as Separable PINN (SPINN), reducing the computational cost from quadratic (or cubic) to linear. We additionally demonstrate that gradient surgery extends to physics-informed Kolmogorov-Arnold Networks (PIKANs), yielding substantial accuracy improvements for 3D domain decomposition and confirming the generality of the proposed approach across network architectures.

Related papers

• GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems [17.161755403211252]
  We introduce the Geometry-Adaptive Integral Autoencoder (GAIA), an operator learning model that encodes the domain boundary and the interior field distribution into geometry tokens.<n>This yields a single architecture for forward (including BVPs) and inverse problems on arbitrary domains in one pass.<n>We evaluate GAIA on seven 2D and 3D benchmarks, four of which are new or substantially extended benchmarks for inverse problems and BVP.
  arXiv  Detail & Related papers  (2026-07-01T16:14:22Z)
• Joint Optimization of Model Partitioning and Resource Allocation for Anti-Jamming Collaborative Inference Systems [52.842088497389746]
  This letter focuses on an anti-jamming collaborative inference system in the presence of a malicious jammer.<n>We first analyze the effects of jamming and DNN partitioning on inference accuracy via data regression.<n>We propose an efficient alternating optimization-based algorithm, which decomposes the problem into three subproblems.
  arXiv  Detail & Related papers  (2026-03-03T03:52:52Z)
• Tail-Aware Post-Training Quantization for 3D Geometry Models [58.79500829118265]
  Post-Training Quantization (PTQ) enables efficient inference without retraining.<n>PTQ fails to transfer effectively to 3D models due to intricate feature distributions and prohibitive calibration overhead.<n>We propose TAPTQ, a Tail-Aware Post-Training Quantization pipeline for 3D geometric learning.
  arXiv  Detail & Related papers  (2026-02-02T07:21:15Z)
• Adaptive Mesh-Quantization for Neural PDE Solvers [51.26961483962011]
  Graph Neural Networks can handle the irregular meshes required for complex geometries and boundary conditions, but still apply uniform computational effort across all nodes.<n>We propose Adaptive Mesh Quantization: spatially adaptive quantization across mesh node, edge, and cluster features, dynamically adjusting the bit-width used by a quantized model.<n>We demonstrate our framework's effectiveness by integrating it with two state-of-the-art models, MP-PDE and GraphViT, to evaluate performance across multiple tasks.
  arXiv  Detail & Related papers  (2025-11-23T14:47:24Z)
• Gated X-TFC: Soft Domain Decomposition for Forward and Inverse Problems in Sharp-Gradient PDEs [0.0]
  Gated X-TFC is a novel framework for both forward and inverse problems.<n>It overcomes limitations through a soft, learned domain decomposition.<n>Gated X-TFC achieves superior accuracy and dramatic improvements in computational efficiency.
  arXiv  Detail & Related papers  (2025-10-01T15:43:02Z)
• Solved in Unit Domain: JacobiNet for Differentiable Coordinate-Transformed PINNs [8.879523745749973]
  JacobiNet is a learning-based coordinate-transformed PINN framework.<n>It mitigates domain mapping and PDE solving within an end-to-end differentiable architecture.<n>It improves prediction accuracy by an average of 3.65*, while delivering over 10* speed up-demonstrating strong generalization, accuracy, and efficiency.
  arXiv  Detail & Related papers  (2025-08-04T15:45:03Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Dual Cone Gradient Descent for Training Physics-Informed Neural Networks [0.0]
  Physics-informed dual neural networks (PINNs) have emerged as a prominent approach for solving partial differential equations.<n>We propose a novel framework, Dual Cone Gradient Descent (DCGD), which adjusts the direction of the updated gradient to ensure it falls within a cone region.
  arXiv  Detail & Related papers  (2024-09-27T03:27:46Z)
• Error Analysis of Three-Layer Neural Network Trained with PGD for Deep Ritz Method [7.723218675113336]
  We employ a three-layer tanh neural network within the framework of the deep Ritz method to solve second-order elliptic equations. We perform projected gradient descent to train the three-layer network and we establish its global convergence. We present error bound in terms of the sample size $n$ and our work provides guidance on how to set the network depth, width, step size, and number of iterations for the projected gradient descent algorithm.
  arXiv  Detail & Related papers  (2024-05-19T05:07:09Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• InverseForm: A Loss Function for Structured Boundary-Aware Segmentation [80.39674800972182]
  We present a novel boundary-aware loss term for semantic segmentation using an inverse-transformation network. This plug-in loss term complements the cross-entropy loss in capturing boundary transformations. We analyze the quantitative and qualitative effects of our loss function on three indoor and outdoor segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-04-06T18:52:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.