Functional Similarity Metric for Neural Networks: Overcoming Parametric Ambiguity via Activation Region Analysis

• URL: http://arxiv.org/abs/2604.16426v1
• Date: Sat, 04 Apr 2026 17:50:07 GMT
• Title: Functional Similarity Metric for Neural Networks: Overcoming Parametric Ambiguity via Activation Region Analysis
• Abstract summary: In ReLU networks, identical functional mappings can be achieved through entirely different weight configurations.<n>Traditional comparison methods exhibit extreme instability to slight weight perturbations during training.<n>This paper proposes a mathematically grounded approach to constructing a stable canonical representation of neural networks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As modern deep learning architectures grow in complexity, representational ambiguity emerges as a critical barrier to their interpretability and reliable merging. For ReLU networks, identical functional mappings can be achieved through entirely different weight configurations due to algebraic symmetries: neuron permutation and positive diagonal scaling. Consequently, traditional parameter-based comparison methods exhibit extreme instability to slight weight perturbations during training. This paper proposes a mathematically grounded approach to constructing a stable canonical representation of neural networks and a robust functional similarity metric. We shift focus from comparing raw weights to analyzing the topology of neuron activation regions. The algorithm first eliminates scaling ambiguity via L2-normalization of weight vectors with subsequent layer compensation. Next, discrete approximations of activation regions are generated as binary functional signatures evaluated over a data sample. To overcome the computational bottleneck of comparing large binary vectors, we adapt Locality-Sensitive Hashing, specifically MinHash, providing a fast and statistically precise approximation of the Jaccard index. The final cross-network neuron matching is formulated as a linear sum assignment problem solved via the Hungarian algorithm. We demonstrate theoretically and experimentally that our metric mitigates the neuron "flickering" effect and exhibits exceptional robustness to minor weight perturbations. This framework provides a solid foundation for model merging, transfer learning, objective assessment during pruning, and Explainable AI paradigms.

Related papers

• Scaling Effects and Uncertainty Quantification in Neural Actor Critic Algorithms [0.0]
  We investigate the neural Actor Critic algorithm using shallow neural networks for both the Actor and Critic models.<n>We compare the convergence properties of the network outputs under various scaling schemes as the network width and the number of training steps tend to infinity.
  arXiv  Detail & Related papers  (2026-01-25T19:06:49Z)
• VIKING: Deep variational inference with stochastic projections [48.946143517489496]
  Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks.<n>We propose a simple variational family that considers two independent linear subspaces of the parameter space.<n>This allows us to build a fully-correlated approximate posterior reflecting the overparametrization.
  arXiv  Detail & Related papers  (2025-10-27T15:38:35Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval.<n>A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed.<n>The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• A Scalable Quantum Neural Network for Approximate SRBB-Based Unitary Synthesis [1.3108652488669736]
  A scalable quantum neural network is introduced as a means to approximate any unitary evolution.<n>The SRBB-based synthesis algorithm is also tested on real hardware.
  arXiv  Detail & Related papers  (2024-12-04T07:21:23Z)
• Concurrent Training and Layer Pruning of Deep Neural Networks [0.0]
  We propose an algorithm capable of identifying and eliminating irrelevant layers of a neural network during the early stages of training. We employ a structure using residual connections around nonlinear network sections that allow the flow of information through the network once a nonlinear section is pruned.
  arXiv  Detail & Related papers  (2024-06-06T23:19:57Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Multilayer Multiset Neuronal Networks -- MMNNs [55.2480439325792]
  The present work describes multilayer multiset neuronal networks incorporating two or more layers of coincidence similarity neurons. The work also explores the utilization of counter-prototype points, which are assigned to the image regions to be avoided.
  arXiv  Detail & Related papers  (2023-08-28T12:55:13Z)
• Interpretable Neural Networks with Random Constructive Algorithm [3.1200894334384954]
  This paper introduces an Interpretable Neural Network (INN) incorporating spatial information to tackle the opaque parameterization process of random weighted neural networks. It devises a geometric relationship strategy using a pool of candidate nodes and established relationships to select node parameters conducive to network convergence.
  arXiv  Detail & Related papers  (2023-07-01T01:07:20Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations [2.047424180164312]
  We study the expectation of a probabilistic neural network as a predictor by itself, focusing on the aggregation of binary activated neural networks with normal distributions over real-valued weights. We show that the exact computation remains tractable for deep but narrow neural networks, thanks to a dynamic programming approach.
  arXiv  Detail & Related papers  (2021-10-28T14:11:07Z)
• Convergence and Alignment of Gradient Descentwith Random Back propagation Weights [6.338178373376447]
  gradient descent with backpropagation is the workhorse of artificial neural networks. Lillicrap et al. propose a more biologically plausible "feedback alignment" algorithm that uses random and fixed backpropagation weights.
  arXiv  Detail & Related papers  (2021-06-10T20:58:05Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.