Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs

• URL: http://arxiv.org/abs/2604.14472v1
• Date: Wed, 15 Apr 2026 23:09:19 GMT
• Title: Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs
• Abstract summary: We study a hybrid design in which the governing PDE residual remains automatic-differentiation based, while finite differences appear only in a weak auxiliary term.<n>The FD term regularizes the residual field without replacing the PDE residual itself.<n>Across seeds 0-5 and 100k epochs, the most reliable tested configuration is a fixed shell weight of 5e-4 under the Kourkoutas-beta regime.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Physics-informed neural networks (PINNs) are often selected by a single scalar loss even when the quantity of interest is more specific. We study a hybrid design in which the governing PDE residual remains automatic-differentiation (AD) based, while finite differences (FD) appear only in a weak auxiliary term that penalizes gradients of the sampled residual field. The FD term regularizes the residual field without replacing the PDE residual itself. We examine this idea in two stages. Stage 1 is a controlled Poisson benchmark comparing a baseline PINN, the FD residual-gradient regularizer, and a matched AD residual-gradient baseline. Stage 2 transfers the same logic to a three-dimensional annular heat-conduction benchmark (PINN3D), where baseline errors concentrate near a wavy outer wall and the auxiliary grid is implemented as a body-fitted shell adjacent to the wall. In Stage 1, the FD regularizer reproduces the main effect of residual-gradient control while exposing a trade-off between field accuracy and residual cleanliness. In Stage 2, the shell regularizer improves the application-facing quantities, namely outer-wall flux and boundary-condition behavior. Across seeds 0-5 and 100k epochs, the most reliable tested configuration is a fixed shell weight of 5e-4 under the Kourkoutas-beta optimizer regime: relative to a matched run without the shell term, it reduces the mean outer-wall BC RMSE from 1.22e-2 to 9.29e-4 and the mean wall-flux RMSE from 9.21e-3 to 9.63e-4. Adam with beta2=0.999 becomes usable when the initial learning rate is reduced to 1e-3, although its shell benefit is less robust than under Kourkoutas-beta. Overall, the results support a targeted view of hybrid PINNs: an auxiliary-only FD regularizer is most valuable when it is aligned with the physical quantity of interest, here the outer-wall flux.

Related papers

• From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts [61.695980784491475]
  Test-time scaling produces cohorts of reasoning rollouts, yet there is no standard label-free account of how their internal computation reorganizes as inference unfolds.<n>We introduce routing effective rank deff, the entropy-effective dimensionality of a cross-rollout graph built from MoE expert-routing similarity.
  arXiv  Detail & Related papers  (2026-09-06T05:43:22Z)
• Generative Nested Sampling of Atomistic Thermodynamic Landscapes [37.547720439639015]
  Flow-based nested sampling has removed the bottleneck for gravitational-wave (GW) inference.<n>We show that LJ disks in PBC reduce energy evaluations by over two orders of magnitude and wall-clock time by roughly one third.<n>For LJ disks in PBC, the algorithm reduces energy evaluations by over two orders of magnitude and wall-clock time by roughly one third.
  arXiv  Detail & Related papers  (2026-09-02T22:21:10Z)
• Batch-wise Adaptive Pruning: Periodic Neuron Activation-Aware Weight Pruning for Language Reasoning Model [41.842373153015586]
  We propose a training-free adaptive pruning method for batched inference in LRMs.<n>We replace threshold-based selection with periodic top-k selection over the aggregated importance scores.<n>Our method outperforms the previous state-of-the-art adaptive pruning method by 39.7 percentage points in average accuracy at batch size 4.
  arXiv  Detail & Related papers  (2026-08-14T06:46:02Z)
• SPARC-Net: A Spectral, Causality-Aware, and Hard-Constrained Physics-Informed Architecture for Stiff and Shock-Dominated Partial Differential Equations [0.0654051537828115]
  Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs)<n>PINNs suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions.<n>We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation.<n>We present-Net, a unified architecture and training
  arXiv  Detail & Related papers  (2026-07-13T09:23:17Z)
• Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance [0.6768558752130311]
  We analyze Diffusion-free guidance (CFG) through an integrator-analysis lens.<n>We show that guidance re-stiffens exactly the discriminative subspace to an anomalous 1+w.<n> DDIM is no longer fitted there, and on coarse meshes its guided residual diverges as sigma_min goes zero.
  arXiv  Detail & Related papers  (2026-07-08T17:21:51Z)
• Kolmogorov Regression for Robust Diffusion Policies [0.0]
  Finite-dimensional (FD) diffusion policies exhibit temporal drift owing to discretization artifacts.<n>We introduce a backward Kolmogorov equation that lifts diffusion policies to a Cameron-Martin space.<n>We certify the dispatch policies with Hamilton-Jacobi reachability theory which reduces deadlock events by 96%.
  arXiv  Detail & Related papers  (2026-06-16T17:18:54Z)
• Constrained Diffusion Models with Primal-Dual Inference [76.64466170898369]
  We develop constrained diffusion models with primal-dual inference (PDI)<n>PDI infers the optimal primal distribution and its parametrizing dual variable.<n>We evaluate PDI on constrained sampling from a mixture of Gaussians, wireless resource allocation, and portfolio management.
  arXiv  Detail & Related papers  (2026-06-15T18:32:33Z)
• When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms [43.8784307709823]
  Long-horizon autoregressive forecasting of physical signals is limited by error accumulation.<n>We ask when such rollout stays stable using synthetic three-component seismograms as a physically structured testbed.
  arXiv  Detail & Related papers  (2026-06-09T13:46:14Z)
• Physics-Informed Residuals for Adaptive Mesh Refinement in Finite-Difference PDE Solvers [0.0]
  This paper studies a hybrid strategy in which a physics-informed neural network (PINN) is used not as the final solver, but as an off-grid residual probe for adaptive mesh refinement.<n>The PINN residual is sampled over the domain, converted into cellwise indicators, and used to guide refinement before the final approximation is computed by a finite-difference solver.
  arXiv  Detail & Related papers  (2026-06-01T16:47:21Z)
• Recovering Physical Dynamics from Discrete Observations via Intrinsic Differential Consistency [12.266189649117003]
  We replace local supervision with a global structural constraint.<n>We train a time-conditioned secant velocity field whose deviation from this property serves two purposes.<n>As a training regularizer, it confines the hypothesis space to flows that compose consistently across temporal scales.
  arXiv  Detail & Related papers  (2026-05-08T20:24:32Z)
• Bias-Corrected Adaptive Conformal Inference for Multi-Horizon Time Series Forecasting [0.0]
  Adaptive Conformal Inference (ACI) provides distribution-free prediction intervals with coverage guarantees for time series under distribution shift.<n>When a base forecaster develops persistent bias after a regime change, ACI compensates by widening intervals symmetrically, producing unnecessarily conservative bands.<n>We propose Bias-Corrected ACI (BC-ACI), which augments standard ACI with an exponentially weighted online moving average (EWM) estimate of forecast bias.
  arXiv  Detail & Related papers  (2026-04-14T19:33:59Z)
• FI-KAN: Fractal Interpolation Kolmogorov-Arnold Networks [0.0]
  Kolmogorov-Arnold Networks (KAN) employ B-spline bases on a fixed grid, providing no intrinsic multi-scale decomposition for non-smooth function approximation.<n>We introduce Fractal Interpolation KAN (FI-KAN), which incorporates learnable fractal function (FIF) bases from iterated function system (IFS) theory into KAN.<n>Two variants are presented: Pure FI-KAN replaces B-splines entirely with FIF bases; Hybrid FI-KAN retains the B-spline path and adds a learnable fractal correction.
  arXiv  Detail & Related papers  (2026-03-30T11:09:07Z)
• Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes [3.305265383862785]
  We present an empirical study of Low-Rank Adaptation (LoRA) in sequential transformer encoder finetuning with companion representation probes.<n>In five full-validation BERT-base reruns on an RTE->MRPC->CoLA->SST-2 sequence, full fine-tuning yields 19.9% +/- average forgetting, whereas standard LoRA (r, query/value modules) yields 0.6% +/-1.4%.
  arXiv  Detail & Related papers  (2026-03-29T14:14:36Z)
• Evaluating Factor-Wise Auxiliary Dynamics Supervision for Latent Structure and Robustness in Simulated Humanoid Locomotion [0.0]
  DynaMITE is a transformer encoder with a factored 24-d latent trained by per-factor auxiliary losses.<n>The supervised latent shows no evidence of decodable or functionally separable factor structure.<n>For locomotion practitioners: auxiliary dynamics supervision does not produce an interpretable estimator.
  arXiv  Detail & Related papers  (2026-03-22T14:56:10Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction [45.25461515976432]
  Plug-and-Play diffusion prior (DP) frameworks have emerged as a powerful paradigm for imaging reconstruction.<n>We present a novel approach to resolving bias-hallucination trade-off, achieving state-of-the-art gradients with significantly accelerated convergence.
  arXiv  Detail & Related papers  (2026-02-26T16:58:43Z)
• Structure-Informed Estimation for Pilot-Limited MIMO Channels via Tensor Decomposition [51.56484100374058]
  This paper formulates pilot-limited channel estimation as low-rank tensor completion from sparse observations.<n>Experiments on synthetic channels demonstrate 10-20,dB normalized mean-square error (NMSE) improvement over least-squares (LS)<n> evaluations on DeepMIMO ray-tracing channels show 24-44% additional NMSE reduction over pure tensor-based methods.
  arXiv  Detail & Related papers  (2026-02-03T23:38:05Z)
• On the Training Instability of Shuffling SGD with Batch Normalization [44.28777474091466]
  Single Shuffle (SS) and Random Reshuffle (RR) interact surprisingly differently in the presence of batch normalization. We show that SS leads to distorted optima in regression and divergence for classification, whereas RR avoids both distortion and divergence.
  arXiv  Detail & Related papers  (2023-02-24T04:10:54Z)
• The KFIoU Loss for Rotated Object Detection [115.334070064346]
  In this paper, we argue that one effective alternative is to devise an approximate loss who can achieve trend-level alignment with SkewIoU loss. Specifically, we model the objects as Gaussian distribution and adopt Kalman filter to inherently mimic the mechanism of SkewIoU. The resulting new loss called KFIoU is easier to implement and works better compared with exact SkewIoU.
  arXiv  Detail & Related papers  (2022-01-29T10:54:57Z)
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times [51.61278695776151]
  Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively. Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
  arXiv  Detail & Related papers  (2021-01-11T19:43:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.