PhysSAE: Mechanistic Interpretability of PINNs with Sparse Autoencoders

• URL: http://arxiv.org/abs/2609.07061v2
• Date: Sat, 12 Sep 2026 19:14:34 GMT
• Title: PhysSAE: Mechanistic Interpretability of PINNs with Sparse Autoencoders
• Abstract summary: PhysSAE is a mechanistic interpretability framework that trains overcomplete sparse autoencoders on PINN penultimate-layer activations.<n>We show that PINNs develop sparse, physically structured latent representations that can be identified and causally interrogated post-hoc.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-Informed Neural Networks (PINNs) embed PDE residuals into neural network training, but their internal representations remain opaque: it is unknown what physical features their hidden layers encode or whether those features have a localized causal role. We present PhysSAE, a mechanistic interpretability framework that trains overcomplete sparse autoencoders (SAEs) on PINN penultimate-layer activations and evaluates dictionary atoms through direct causal intervention in the original frozen hidden state: $h_{\mathrm{cf}} = h - αz_k d_k$, bypassing the SAE decoder entirely. Across six PDE families, with 3 PINN seeds and 3 SAE seeds each---we show that (i) Our discovered SAE atoms align with independently-defined physical observables (max Pearson $|r|=0.951$, always $\gg$ permutation null), (ii) the causal footprint of top-aligned atom ablation is 1.2--4.2$\times$ more spatially concentrated canonical than PCA or ICA interventions, and (iii) top-aligned atoms outperform matched random controls on causal localization for structured physical concepts (ESF$_{80}$ advantage 0.04-0.44). Two-atom bilateral representations improve concept regression R$^2$ by $ΔR^2\!=\!0.05\text{-}0.15$ over single atoms, while random pairs decrease it by up to 0.60. These results demonstrate that PINNs develop sparse, physically structured latent representations that can be identified and causally interrogated post-hoc, opening a path toward interpretability-aware scientific machine learning.

Related papers

• A variational physics-informed graph neural network for heterogeneous solid mechanics [0.0]
  This work presents a variational, label-free physics-informed graph neural network (PI-GNN) in which the conforming is carried by the discretization rather than by the trial field.
  arXiv  Detail & Related papers  (2026-09-10T02:02:32Z)
• Error Highways: Scaling Predictive Coding to Very Deep Networks [45.88028371034407]
  Predictive coding networks (PCNs) offer a biologically-plausible, local-learning alternative to back-propagation of errors (backprop)<n>A central obstacle to scaling PCNs is that the learning signal decays rapidly as it propagates away from clamped boundaries.<n>We propose highway error propagation (HEP), a scheme that augments the free energy function underlying predictive coding (PC) by altering its neural structure.
  arXiv  Detail & Related papers  (2026-06-22T01:12:29Z)
• Safety, Security, and Cognitive Risks in Neuro-Symbolic AI [0.0]
  Neuro-symbolic AI (NeSy) pairs neural perception with symbolic reasoning.<n>NeSy attacks five layers: neural perception, symbolic knowledge bases, reasoning engines, agentic orchestration, and data stores.
  arXiv  Detail & Related papers  (2026-06-15T19:11:55Z)
• Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography [13.891522069967507]
  We show that semantic features alone recover 94% of peak encoding performance.<n>We also show that SAE-discovered features recapitulate known cortical semantic organization.<n>Results generalize across English, Chinese, and French.
  arXiv  Detail & Related papers  (2026-05-21T21:00:24Z)
• Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse [7.167095059974211]
  This work investigates feature survival under severe capacity constraints in Variational Autoencoder--Sparse Autoencoder architectures.<n>We introduce an adaptive sparsity scheduling framework that progressively reduces active neurons from 500 to 50 over 50 training epochs.<n>Our key finding reveals a pervasive paradox: while global representation quality remains stable, local feature interpretability collapses systematically.
  arXiv  Detail & Related papers  (2026-03-18T00:16:38Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• Comprehensive Validation of Replica Symmetry Breaking via Quantum Annealing: From Ground States to Topological Collapse [0.0]
  We extend Giorgio Parisi's exact solution of the Sherrington-Kirkpatrick spin glass to 4000 spins.<n>We probe both the emergence and breakdown of replica symmetry breaking.<n>This comprehensive validation establishes quantum advantage for probing fundamental statistical mechanics in complex systems.
  arXiv  Detail & Related papers  (2025-11-09T14:33:22Z)
• Extrapolation to infinite model space of no-core shell model calculations using machine learning [41.99844472131922]
  An ensemble of neural networks is employed to extrapolate no-core shell model (NCSM) results to infinite model space for light nuclei.<n>We present a review of our neural network extrapolations of the NCSM results obtained with the Daejeon16 NN interaction in different model spaces.
  arXiv  Detail & Related papers  (2025-11-07T08:16:12Z)
• v-PuNNs: van der Put Neural Networks for Transparent Ultrametric Representation Learning [0.0]
  We introduce van der Put Neural Networks (v-PuNNs), the first architecture whose neurons are characteristic functions of p-adic balls in $mathbbZ_p$.<n>Under our Transparent Ultrametric Representation Learning (TURL) principle every weight is itself a p-adic number, giving exact subtree semantics.<n>V-PuNNs therefore bridge number theory and deep learning, offering exact, interpretable, and efficient models for hierarchical data.
  arXiv  Detail & Related papers  (2025-08-01T18:23:38Z)
• Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations [48.46721044282335]
  tensor decomposition networks (TDNs) achieve competitive performance with dramatic speedup in computations.<n>We evaluate TDNs on PubChemQCR, a newly curated molecular relaxation dataset containing 105 million DFT-calculated snapshots.<n>Results show that TDNs achieve competitive performance with dramatic speedup in computations.
  arXiv  Detail & Related papers  (2025-07-01T18:46:27Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs [42.551773746803946]
  Vision tasks are characterized by the properties of locality and translation invariance. The superior performance of convolutional neural networks (CNNs) on these tasks is widely attributed to the inductive bias of locality and weight sharing baked into their architecture. Existing attempts to quantify the statistical benefits of these biases in CNNs over locally connected neural networks (LCNs) and fully connected neural networks (FCNs) fall into one of the following categories.
  arXiv  Detail & Related papers  (2024-03-23T03:57:28Z)
• Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds [75.51968172401394]
  We study the sample complexity of the neural policy mirror descent (NPMD) algorithm with deep convolutional neural networks (CNN) In each iteration of NPMD, both the value function and the policy can be well approximated by CNNs. We show that NPMD can leverage the low-dimensional structure of state space to escape from the curse of dimensionality.
  arXiv  Detail & Related papers  (2023-09-25T07:31:22Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
  Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
  arXiv  Detail & Related papers  (2021-06-06T19:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.