Interpreting FCDNNs via RG on Exponential Family

• URL: http://arxiv.org/abs/2606.00157v1
• Date: Fri, 29 May 2026 07:29:08 GMT
• Title: Interpreting FCDNNs via RG on Exponential Family
• Abstract summary: We establish the interpretability theory of deep learning through constructing a relationship between the renormalization group (RG) method and the training process of deep neural networks (DNNs)<n>We generalize our results to the case of continuous input data, which is a necessary preparation for applying the corresponding framework to real-world data.<n>This conclusion shows that the training process of DNNs is equivalent to RG calculation on this kind of data and therefore the network can extract main features from the input data just like RG.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider establishing the interpretability theory of deep learning through constructing a corresponding relationship between the renormalization group (RG) method in statistical physics and the training process of deep neural networks (DNNs). We have proved the constructed relationship using the one-dimensional Ising model as the input data. In this paper we generalize our results to the case of continuous input data, which is a necessary preparation for applying the corresponding framework to real-world data. To be representative, we consider a class of data distribution in the exponential family. We prove that when the parameters of fully connected (FC) DNNs achieve their optimal value after training, the characteristic parameters of the feature layer output of DNNs are equal to the fixed points of the characteristic parameters of input data under RG method for continuous fields. This conclusion shows that the training process of DNNs is equivalent to RG calculation on this kind of data and therefore the network can extract main features from the input data just like RG. Also, the equivalence further validates the correspondence framework we have established, providing an explanation for the outstanding performance of DNNs on real-world data.

Related papers

• RelGNN: Composite Message Passing for Relational Deep Learning [56.48834369525997]
  We introduce RelGNN, a novel GNN framework specifically designed to leverage the unique structural characteristics of the graphs built from relational databases.<n>RelGNN is evaluated on 30 diverse real-world tasks from Relbench (Fey et al., 2024), and achieves state-of-the-art performance on the vast majority tasks, with improvements of up to 25%.
  arXiv  Detail & Related papers  (2025-02-10T18:58:40Z)
• Mean flow data assimilation using physics-constrained Graph Neural Networks [0.0]
  This study introduces a novel data assimilation approach that integrates Graph Neural Networks (GNNs) with optimisation techniques to enhance the accuracy of mean flow reconstruction.<n>The GNN framework is well-suited for handling unstructured data, which is common in the complex geometries encountered in Computational Fluid Dynamics (CFD)<n>Results demonstrate significant improvements in the accuracy of mean flow reconstructions, even with limited training data, compared to analogous purely data-driven models.
  arXiv  Detail & Related papers  (2024-11-14T14:31:52Z)
• Deep Learning as Ricci Flow [38.27936710747996]
  Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. We show that the transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.
  arXiv  Detail & Related papers  (2024-04-22T15:12:47Z)
• Interpretable A-posteriori Error Indication for Graph Neural Network Surrogate Models [0.0]
  This work introduces an interpretability enhancement procedure for graph neural networks (GNNs) The end result is an interpretable GNN model that isolates regions in physical space, corresponding to sub-graphs, that are intrinsically linked to the forecasting task. The interpretable GNNs can also be used to identify, during inference, graph nodes that correspond to a majority of the anticipated forecasting error.
  arXiv  Detail & Related papers  (2023-11-13T18:37:07Z)
• Neural Tangent Kernels Motivate Graph Neural Networks with Cross-Covariance Graphs [94.44374472696272]
  We investigate NTKs and alignment in the context of graph neural networks (GNNs) Our results establish the theoretical guarantees on the optimality of the alignment for a two-layer GNN. These guarantees are characterized by the graph shift operator being a function of the cross-covariance between the input and the output data.
  arXiv  Detail & Related papers  (2023-10-16T19:54:21Z)
• Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias [72.33336385797944]
  We propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias.<n>We show that LD significantly outperforms state-of-the-art methods on Open Graph Benchmark datasets.
  arXiv  Detail & Related papers  (2023-09-26T13:09:43Z)
• Deep neural networks have an inbuilt Occam's razor [4.330196787806085]
  We show that structured data combined with an intrinsic Occam's razor-like inductive bias towards simple functions counteracts the exponential growth of functions with complexity.<n>This analysis reveals that structured data, combined with an intrinsic Occam's razor-like inductive bias towards (Kolmogorov) simple functions that is strong enough to counteract the exponential growth of functions with complexity, is a key to the success of DNNs.
  arXiv  Detail & Related papers  (2023-04-13T16:58:21Z)
• Explicit Context Integrated Recurrent Neural Network for Sensor Data Applications [0.0]
  Context Integrated RNN (CiRNN) enables integrating explicit contexts represented in the form of contextual features. Experiments show an improvement of 39% and 87% respectively, over state-of-the-art models.
  arXiv  Detail & Related papers  (2023-01-12T13:58:56Z)
• coVariance Neural Networks [119.45320143101381]
  Graph neural networks (GNN) are an effective framework that exploit inter-relationships within graph-structured data for learning. We propose a GNN architecture, called coVariance neural network (VNN), that operates on sample covariance matrices as graphs. We show that VNN performance is indeed more stable than PCA-based statistical approaches.
  arXiv  Detail & Related papers  (2022-05-31T15:04:43Z)
• Deep Neural Network Classifier for Multi-dimensional Functional Data [4.340040784481499]
  We propose a new approach, called as functional deep neural network (FDNN), for classifying multi-dimensional functional data. Specifically, a deep neural network is trained based on the principle components of the training data which shall be used to predict the class label of a future data function.
  arXiv  Detail & Related papers  (2022-05-17T19:22:48Z)
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data [52.771780951404565]
  Shift-Robust GNN (SR-GNN) is designed to account for distributional differences between biased training data and the graph's true inference distribution. We show that SR-GNN outperforms other GNN baselines by accuracy, eliminating at least (40%) of the negative effects introduced by biased training data.
  arXiv  Detail & Related papers  (2021-08-02T18:00:38Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• Bayesian Graph Neural Networks with Adaptive Connection Sampling [62.51689735630133]
  We propose a unified framework for adaptive connection sampling in graph neural networks (GNNs) The proposed framework not only alleviates over-smoothing and over-fitting tendencies of deep GNNs, but also enables learning with uncertainty in graph analytic tasks with GNNs.
  arXiv  Detail & Related papers  (2020-06-07T07:06:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.