Emergence of the SVD as an interpretable factorization in deep learning
for inverse problems
- URL: http://arxiv.org/abs/2301.07820v2
- Date: Wed, 9 Aug 2023 00:44:38 GMT
- Title: Emergence of the SVD as an interpretable factorization in deep learning
for inverse problems
- Authors: Shashank Sule, Richard G. Spencer and Wojciech Czaja
- Abstract summary: We demonstrate the emergence of the singular value decomposition (SVD) of the weight matrix as a tool for interpretation of neural networks.
We show that descrambling transformations can be expressed in terms of the SVD of the NN weights and the input autocorrelation matrix.
- Score: 1.5567671045891203
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Within the framework of deep learning we demonstrate the emergence of the
singular value decomposition (SVD) of the weight matrix as a tool for
interpretation of neural networks (NN) when combined with the descrambling
transformation--a recently-developed technique for addressing interpretability
in noisy parameter estimation neural networks \cite{amey2021neural}. By
considering the averaging effect of the data passed to the descrambling
minimization problem, we show that descrambling transformations--in the large
data limit--can be expressed in terms of the SVD of the NN weights and the
input autocorrelation matrix. Using this fact, we show that within the class of
noisy parameter estimation problems the SVD may be the structure through which
trained networks encode a signal model. We substantiate our theoretical
findings with empirical evidence from both linear and non-linear signal models.
Our results also illuminate the connections between a mathematical theory of
semantic development \cite{saxe2019mathematical} and neural network
interpretability.
Related papers
- Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
We propose leveraging the principle of chunking to interpret artificial neural population activities.
We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities.
We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
arXiv Detail & Related papers (2025-02-03T20:30:46Z) - Understanding Artificial Neural Network's Behavior from Neuron Activation Perspective [8.251799609350725]
This paper explores the intricate behavior of deep neural networks (DNNs) through the lens of neuron activation dynamics.
We propose a probabilistic framework that can analyze models' neuron activation patterns as a process.
arXiv Detail & Related papers (2024-12-24T01:01:06Z) - Understanding polysemanticity in neural networks through coding theory [0.8702432681310401]
We propose a novel practical approach to network interpretability and theoretical insights into polysemanticity and the density of codes.
We show how random projections can reveal whether a network exhibits a smooth or non-differentiable code and hence how interpretable the code is.
Our approach advances the pursuit of interpretability in neural networks, providing insights into their underlying structure and suggesting new avenues for circuit-level interpretability.
arXiv Detail & Related papers (2024-01-31T16:31:54Z) - Graph Neural Networks Provably Benefit from Structural Information: A
Feature Learning Perspective [53.999128831324576]
Graph neural networks (GNNs) have pioneered advancements in graph representation learning.
This study investigates the role of graph convolution within the context of feature learning theory.
arXiv Detail & Related papers (2023-06-24T10:21:11Z) - Transferability of coVariance Neural Networks and Application to
Interpretable Brain Age Prediction using Anatomical Features [119.45320143101381]
Graph convolutional networks (GCN) leverage topology-driven graph convolutional operations to combine information across the graph for inference tasks.
We have studied GCNs with covariance matrices as graphs in the form of coVariance neural networks (VNNs)
VNNs inherit the scale-free data processing architecture from GCNs and here, we show that VNNs exhibit transferability of performance over datasets whose covariance matrices converge to a limit object.
arXiv Detail & Related papers (2023-05-02T22:15:54Z) - Neural networks trained with SGD learn distributions of increasing
complexity [78.30235086565388]
We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics.
We then exploit higher-order statistics only later during training.
We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
arXiv Detail & Related papers (2022-11-21T15:27:22Z) - Seeking Interpretability and Explainability in Binary Activated Neural Networks [2.828173677501078]
We study the use of binary activated neural networks as interpretable and explainable predictors in the context of regression tasks.
We present an approach based on the efficient computation of SHAP values for quantifying the relative importance of the features, hidden neurons and even weights.
arXiv Detail & Related papers (2022-09-07T20:11:17Z) - Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs.
By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
arXiv Detail & Related papers (2022-02-01T17:11:13Z) - A neural anisotropic view of underspecification in deep learning [60.119023683371736]
We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation.
Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
arXiv Detail & Related papers (2021-04-29T14:31:09Z) - Topological obstructions in neural networks learning [67.8848058842671]
We study global properties of the loss gradient function flow.
We use topological data analysis of the loss function and its Morse complex to relate local behavior along gradient trajectories with global properties of the loss surface.
arXiv Detail & Related papers (2020-12-31T18:53:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.