The Neural Echo: A Signal Processing Perspective for Understanding Neural Networks

• URL: http://arxiv.org/abs/2608.04864v1
• Date: Wed, 05 Aug 2026 13:57:16 GMT
• Title: The Neural Echo: A Signal Processing Perspective for Understanding Neural Networks
• Abstract summary: We introduce the neural echo as a tool for understanding the behavior of neural networks.<n>It generalizes the model-based concepts of impulse responses, diffusion echoes, and filter echoes to learning-based methods.<n>It provides local, space-adaptive impulse responses and filter kernels for a neural network, its so-called echoes.
• Score: 3.534869097377701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce the neural echo as a tool for understanding the behavior of neural networks. It generalizes the model-based concepts of impulse responses, diffusion echoes, and filter echoes to learning-based methods. It provides local, space-adaptive impulse responses and filter kernels for a neural network, its so-called echoes. These echoes depend on the input image and can be visualized to understand the learned dynamics of the network via an affine mapping. Neural echoes build a bridge from classical signal processing to modern explainable AI. They are very general and can be applied to both image-to-image and classification networks, with convolutional or fully connected structure, of feedforward or recurrent type, including modern transformer networks. Network differentiability is not required. In the differentiable case, neural echoes comprise concepts based on the network Jacobian, such as saliency maps and the analysis of adversarial perturbations, as special instances. As a simple blueprint to explain our framework, we derive neural echoes for the denoising convolutional neural network (DnCNN). Our experiments suggest that this network weights pixels based on their spatial and gray value distances. This not only clarifies its behavior, but also shows that it can reproduce key concepts of classical model-based denoisers such as bilateral filtering.

Related papers

• Semantic representations emerge in biologically inspired ensembles of cross-supervising neural networks [1.5346678870160888]
  We present a model of representation learning by ensembles of neural networks.<n>Each network learns to encode stimuli into an abstract representation space by cross-supervising interactions with other networks.<n>We find that performance is optimal for small receptive fields, and that sparse connectivity between networks is nearly as accurate as all-to-all interactions.
  arXiv  Detail & Related papers  (2025-10-16T09:30:22Z)
• Concept-Guided Interpretability via Neural Chunking [64.6429903327095]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract recurring chunks on a neural population level.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
  We propose leveraging the principle of chunking to interpret artificial neural population activities.<n>We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities.<n>We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
  arXiv  Detail & Related papers  (2025-02-03T20:30:46Z)
• Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks [0.0]
  We introduce and evaluate a brain-like neural network model capable of unsupervised representation learning.<n>The model was tested on a diverse set of popular machine learning benchmarks.
  arXiv  Detail & Related papers  (2024-06-07T08:32:30Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Closed-Form Interpretation of Neural Network Classifiers with Symbolic Gradients [0.7832189413179361]
  I introduce a unified framework for finding a closed-form interpretation of any single neuron in an artificial neural network. I demonstrate how to interpret neural network classifiers to reveal closed-form expressions of the concepts encoded in their decision boundaries.
  arXiv  Detail & Related papers  (2024-01-10T07:47:42Z)
• Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth [92.25666446274188]
  sinusoidal neural networks with activations have been proposed as an alternative to networks with traditional activation functions. We first propose a simplified version of such sinusoidal neural networks, which allows both for easier practical implementation and simpler theoretical analysis. We then analyze the behavior of these networks from the neural tangent kernel perspective and demonstrate that their kernel approximates a low-pass filter with an adjustable bandwidth.
  arXiv  Detail & Related papers  (2022-11-26T07:41:48Z)
• Gaussian Process Surrogate Models for Neural Networks [6.8304779077042515]
  In science and engineering, modeling is a methodology used to understand complex systems whose internal processes are opaque. We construct a class of surrogate models for neural networks using Gaussian processes. We demonstrate our approach captures existing phenomena related to the spectral bias of neural networks, and then show that our surrogate models can be used to solve practical problems.
  arXiv  Detail & Related papers  (2022-08-11T20:17:02Z)
• How and what to learn:The modes of machine learning [7.085027463060304]
  We propose a new approach, namely the weight pathway analysis (WPA), to study the mechanism of multilayer neural networks. WPA shows that a neural network stores and utilizes information in a "holographic" way, that is, the network encodes all training samples in a coherent structure. It is found that hidden-layer neurons self-organize into different classes in the later stages of the learning process.
  arXiv  Detail & Related papers  (2022-02-28T14:39:06Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Visual Pattern Recognition with on On-chip Learning: towards a Fully Neuromorphic Approach [10.181725314550823]
  We present a spiking neural network (SNN) for visual pattern recognition with on-chip learning on neuromorphic hardware. We show how this network can learn simple visual patterns composed of horizontal and vertical bars sensed by a Dynamic Vision Sensor. During recognition, the network classifies the pattern's identity while at the same time estimating its location and scale.
  arXiv  Detail & Related papers  (2020-08-08T08:07:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.