An analytic theory of convolutional neural network inverse problems solvers

• URL: http://arxiv.org/abs/2601.10334v1
• Date: Thu, 15 Jan 2026 12:25:59 GMT
• Title: An analytic theory of convolutional neural network inverse problems solvers
• Authors: Minh Hai Nguyen, Quoc Bao Do, Edouard Pauwels, Pierre Weiss,
• Abstract summary: We analyze trained neural networks through the lens of the Minimum Mean Square Error (MMSE) estimator.<n>Under the empirical training distribution, we derive an analytic, interpretable, and tractable formula for this constrained variant.<n>We demonstrate that our theory matches the neural networks outputs (PSNR $gtrsim25$dB)
• Score: 5.55979411072702
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Supervised convolutional neural networks (CNNs) are widely used to solve imaging inverse problems, achieving state-of-the-art performance in numerous applications. However, despite their empirical success, these methods are poorly understood from a theoretical perspective and often treated as black boxes. To bridge this gap, we analyze trained neural networks through the lens of the Minimum Mean Square Error (MMSE) estimator, incorporating functional constraints that capture two fundamental inductive biases of CNNs: translation equivariance and locality via finite receptive fields. Under the empirical training distribution, we derive an analytic, interpretable, and tractable formula for this constrained variant, termed Local-Equivariant MMSE (LE-MMSE). Through extensive numerical experiments across various inverse problems (denoising, inpainting, deconvolution), datasets (FFHQ, CIFAR-10, FashionMNIST), and architectures (U-Net, ResNet, PatchMLP), we demonstrate that our theory matches the neural networks outputs (PSNR $\gtrsim25$dB). Furthermore, we provide insights into the differences between \emph{physics-aware} and \emph{physics-agnostic} estimators, the impact of high-density regions in the training (patch) distribution, and the influence of other factors (dataset size, patch size, etc).

Related papers

• Uncertainty propagation in feed-forward neural network models [3.987067170467799]
  We develop new uncertainty propagation methods for feed-forward neural network architectures.<n>We derive analytical expressions for the probability density function (PDF) of the neural network output.<n>A key finding is that an appropriate linearization of the leaky ReLU activation function yields accurate statistical results.
  arXiv  Detail & Related papers  (2025-03-27T00:16:36Z)
• Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization [0.18641315013048293]
  Vision Transformers (ViTs) have emerged as a powerful class of models in the field of deep learning for image classification.<n>We propose a novel Random Matrix Theory (RMT)-based method for pruning pre-trained DNNs, based on the sparsification of weights and singular vectors.<n>We demonstrate that our RMT-based pruning can be used to reduce the number of parameters of ViT models by 30-50% with less than 1% loss in accuracy.
  arXiv  Detail & Related papers  (2025-03-02T05:25:20Z)
• Diagonal Symmetrization of Neural Network Solvers for the Many-Electron Schrödinger Equation [10.862915635270346]
  We study different ways of incorporating diagonal invariance in neural network ans"atze trained via variational Monte Carlo methods.<n>We show that, contrary to standard ML setups, in-training symmetrization destabilizes training and can lead to worse performance.<n>Our theoretical and numerical results indicate that this unexpected behavior may arise from a unique computational-statistical tradeoff not found in standard ML analyses of symmetrization.
  arXiv  Detail & Related papers  (2025-02-07T20:37:25Z)
• ASPINN: An asymptotic strategy for solving singularly perturbed differential equations [12.14934707131722]
  We propose Asymptotic Physics-Informed Neural Networks (ASPINN), a generalization of Physics-Informed Neural Networks (PINN) and General-Kindred Physics-Informed Neural Networks (GKPINN) ASPINN has a strong fitting ability for solving SPDEs due to the placement of exponential layers at the boundary layer. We demonstrate the effect of ASPINN by solving diverse classes of SPDEs, which clearly shows that the ASPINN method is promising in boundary layer problems.
  arXiv  Detail & Related papers  (2024-09-20T03:25:17Z)
• Feature Mapping in Physics-Informed Neural Networks (PINNs) [1.9819034119774483]
  We study the training dynamics of PINNs with a feature mapping layer via the limiting Conjugate Kernel and Neural Tangent Kernel. We propose conditionally positive definite Radial Basis Function as a better alternative.
  arXiv  Detail & Related papers  (2024-02-10T13:51:09Z)
• SO(2) and O(2) Equivariance in Image Recognition with Bessel-Convolutional Neural Networks [63.24965775030674]
  This work presents the development of Bessel-convolutional neural networks (B-CNNs) B-CNNs exploit a particular decomposition based on Bessel functions to modify the key operation between images and filters. Study is carried out to assess the performances of B-CNNs compared to other methods.
  arXiv  Detail & Related papers  (2023-04-18T18:06:35Z)
• Learning Discretized Neural Networks under Ricci Flow [48.47315844022283]
  We study Discretized Neural Networks (DNNs) composed of low-precision weights and activations.<n>DNNs suffer from either infinite or zero gradients due to the non-differentiable discrete function during training.
  arXiv  Detail & Related papers  (2023-02-07T10:51:53Z)
• On the limits of neural network explainability via descrambling [2.5554069583567487]
  We show that the principal components of the hidden layer preactivations can be characterized as the optimal explainers or descramblers for the layer weights. We show that in typical deep learning contexts these descramblers take diverse and interesting forms.
  arXiv  Detail & Related papers  (2023-01-18T23:16:53Z)
• Variational Neural Networks [88.24021148516319]
  We propose a method for uncertainty estimation in neural networks called Variational Neural Network (VNN) VNN generates parameters for the output distribution of a layer by transforming its inputs with learnable sub-layers. In uncertainty quality estimation experiments, we show that VNNs achieve better uncertainty quality than Monte Carlo Dropout or Bayes By Backpropagation methods.
  arXiv  Detail & Related papers  (2022-07-04T15:41:02Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Robust Learning of Physics Informed Neural Networks [2.86989372262348]
  Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE.
  arXiv  Detail & Related papers  (2021-10-26T00:10:57Z)
• Physics informed neural networks for continuum micromechanics [68.8204255655161]
  Recently, physics informed neural networks have successfully been applied to a broad variety of problems in applied mathematics and engineering. Due to the global approximation, physics informed neural networks have difficulties in displaying localized effects and strong non-linear solutions by optimization. It is shown, that the domain decomposition approach is able to accurately resolve nonlinear stress, displacement and energy fields in heterogeneous microstructures obtained from real-world $mu$CT-scans.
  arXiv  Detail & Related papers  (2021-10-14T14:05:19Z)
• Influence Estimation and Maximization via Neural Mean-Field Dynamics [60.91291234832546]
  We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems. Our framework can simultaneously learn the structure of the diffusion network and the evolution of node infection probabilities.
  arXiv  Detail & Related papers  (2021-06-03T00:02:05Z)
• How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks [80.55378250013496]
  We study how neural networks trained by gradient descent extrapolate what they learn outside the support of the training distribution. Graph Neural Networks (GNNs) have shown some success in more complex tasks.
  arXiv  Detail & Related papers  (2020-09-24T17:48:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.