Modularity-Free Conflict-Averse Training for Generalized PINNs

• URL: http://arxiv.org/abs/2606.20156v1
• Date: Thu, 18 Jun 2026 12:20:29 GMT
• Title: Modularity-Free Conflict-Averse Training for Generalized PINNs
• Abstract summary: Physics-informed neural networks (PINNs) have become a powerful framework for solving PDEs.<n>We show that training PINNs remains fragile as their effectiveness deteriorates as model capacity increases.<n>We propose a novel framework, Modular-Sparsity Synchronization (ModSync), which integrates structural optimization into conflict-averse training.
• Score: 46.12829178558227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physics-informed neural networks (PINNs) have become a powerful framework for solving PDEs by embedding physical laws into differentiable objectives. Despite their advances, training PINNs remains fragile: recent conflict-averse optimization schemes alleviate gradient interference between residual and boundary losses, but we show that their effectiveness deteriorates as model capacity increases. In this paper, we identify a capacity-induced failure mode, where overparameterized networks undergo functional modularity, self-partitioning into task-exclusive modules that suppress cross-objective interaction and hinder convergence toward Pareto-stationary points. To address this issue, we propose a novel framework, Modular-Sparsity Synchronization (ModSync), which integrates structural optimization into conflict-averse training by penalizing task-exclusive connections while preserving interaction-promoting pathways. Extensive experiments across diverse PDE benchmarks demonstrate that ModSync consistently prevents capacity-driven failures, sustains robust cross-objective coupling, and achieves state-of-the-art accuracy. Codes are available at \url{https://github.com/heejokong/ModSync}.

Related papers

• Unlocking the Potential of Continual Model Merging: An ODE Perspective [0.42970700836450487]
  Continual Model Merging (CMM) enables rapid customization of foundation models by sequentially incorporating task-adapted models without repeated retraining.<n>We propose ODE-driven Merging (ODE-M), a controllable framework that formulates each merge as a trajectory in parameter space rather than a one-step endpoint update.
  arXiv  Detail & Related papers  (2026-05-19T06:03:20Z)
• Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning [57.10440766103372]
  Trajectory Regularized Merging (TRM) is a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace.<n>Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics.
  arXiv  Detail & Related papers  (2026-05-08T14:07:32Z)
• Forked Physics Informed Neural Networks for Coupled Systems of Differential equations [1.7158443622461192]
  We propose a Forked PINN framework designed for coupled systems of differential equations (DEs)<n>We demonstrate the effectiveness of FPINN in simulating non-Markovian open quantum dynamics governed by coupled DEs.<n>For the spin-boson and XXZ models, FPINN accurately captures non-Markovian features, such as quantum coherence revival and information backflow.
  arXiv  Detail & Related papers  (2026-02-16T08:32:59Z)
• SpanNorm: Reconciling Training Stability and Performance in Deep Transformers [55.100133502295996]
  We propose SpanNorm, a novel technique designed to resolve the dilemma by integrating the strengths of both paradigms.<n>We provide a theoretical analysis demonstrating that SpanNorm, combined with a principled scaling strategy, maintains bounded signal variance throughout the network.<n> Empirically, SpanNorm consistently outperforms standard normalization schemes in both dense and Mixture-of-Experts (MoE) scenarios.
  arXiv  Detail & Related papers  (2026-01-30T05:21:57Z)
• Architecture-Optimization Co-Design for Physics-Informed Neural Networks Via Attentive Representations and Conflict-Resolved Gradients [5.447935819547941]
  We study PINN training from a unified architecture-optimization perspective.<n>We propose a layer-wise dynamic attention mechanism to enhance representational flexibility.<n>We then reformulate PINN training as a multi-task learning problem and introduce a conflict-resolved gradient update strategy.
  arXiv  Detail & Related papers  (2026-01-19T11:32:25Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• AutoBalance: An Automatic Balancing Framework for Training Physics-Informed Neural Networks [10.223108587188808]
  PINNs provide a powerful and general framework for solving Partial Differential Equations.<n> PINNs balance multiple loss terms, such as PDE residuals and boundary conditions, which often have conflicting objectives and vastly different curvatures.<n>Existing methods address this issue by manipulating gradients before optimization.<n>We introduce AutoBalance, a novel "post-combine" training paradigm.
  arXiv  Detail & Related papers  (2025-10-08T06:13:03Z)
• Modular Delta Merging with Orthogonal Constraints: A Scalable Framework for Continual and Reversible Model Composition [0.0]
  Existing approaches to model merging and continual learning often suffer from task interference, catastrophic forgetting, or lack of reversibility.<n>We propose Modular Delta Merging with Orthogonal Constraints (MDM-OC), a novel framework that enables scalable, interference-free, and composition of fine-tuned models.
  arXiv  Detail & Related papers  (2025-07-28T17:08:49Z)
• Mask-PINNs: Mitigating Internal Covariate Shift in Physics-Informed Neural Networks [1.2667864219315372]
  PINNs have emerged as a powerful framework for solving partial differential equations.<n>We propose Mask-PINNs, a learnable mask function to regulate feature distributions.<n>Our results show consistent improvements in prediction accuracy, convergence stability, and robustness.
  arXiv  Detail & Related papers  (2025-05-09T15:38:52Z)
• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.