Neural Network Compression by Approximate Differential Equivalence

• URL: http://arxiv.org/abs/2606.01402v1
• Date: Sun, 31 May 2026 19:01:46 GMT
• Title: Neural Network Compression by Approximate Differential Equivalence
• Abstract summary: We propose an approach that compresses models by aggregating neurons rather than removing weights independently.<n>Our method encodes a trained network as induces a smooth ODE system and applies a lumping method called Approximate Forward Differential Equivalence.<n>We evaluate the method on synthetic datasets derived from nonlinear systems with known ground-truth behavior and on public regression benchmarks.
• Score: 0.9099663022952496
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Neural network compression is commonly achieved by pruning parameters based on local importance scores, e.g., magnitude-based pruning. We propose a complementary approach that compresses models by aggregating neurons with similar functional behavior rather than removing weights independently. Our method encodes a trained network as a polynomial ODE system and applies a lumping method called Approximate Forward Differential Equivalence to identify neurons with approximately matching induced dynamics. A single tolerance parameter, $\varepsilon$, controls the compression level and induces a smooth trade-off between model size and predictive accuracy. We evaluate the method on synthetic datasets derived from nonlinear dynamical systems with known ground-truth behavior and on public regression benchmarks. Across both settings, the proposed approach achieves substantial parameter reduction while preserving accuracy, and consistently compares favorably with magnitude-based pruning and Wanda at similar compression levels. These results suggest that differential equivalence-based aggregation is a principled and effective alternative to conventional weight-centric pruning.

Related papers

• Robust Basis Spline Decoupling for the Compression of Transformer Models [1.1470070927586018]
  We introduce a B-spline-based decoupling framework that generalizes existing approaches.<n>By exploiting the local support and flexible smoothness control of B-splines, the proposed formulation yields a more stable and expressive representation.
  arXiv  Detail & Related papers  (2026-05-11T06:37:36Z)
• Functional Similarity Metric for Neural Networks: Overcoming Parametric Ambiguity via Activation Region Analysis [0.0]
  In ReLU networks, identical functional mappings can be achieved through entirely different weight configurations.<n>Traditional comparison methods exhibit extreme instability to slight weight perturbations during training.<n>This paper proposes a mathematically grounded approach to constructing a stable canonical representation of neural networks.
  arXiv  Detail & Related papers  (2026-04-04T17:50:07Z)
• Certified Neural Approximations of Nonlinear Dynamics [51.01318247729693]
  In safety-critical contexts, the use of neural approximations requires formal bounds on their closeness to the underlying system.<n>We propose a novel, adaptive, and parallelizable verification method based on certified first-order models.
  arXiv  Detail & Related papers  (2025-05-21T13:22:20Z)
• Convolutional Neural Network Compression Based on Low-Rank Decomposition [3.3295360710329738]
  This paper proposes a model compression method that integrates Variational Bayesian Matrix Factorization. VBMF is employed to estimate the rank of the weight tensor at each layer. Experimental results show that for both high and low compression ratios, our compression model exhibits advanced performance.
  arXiv  Detail & Related papers  (2024-08-29T06:40:34Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression [63.56922682378755]
  We focus on extending spatial aggregation capability and propose a dynamic kernel-based transform coding. The proposed adaptive aggregation generates kernel offsets to capture valid information in the content-conditioned range to help transform. Experimental results demonstrate that our method achieves superior rate-distortion performance on three benchmarks compared to the state-of-the-art learning-based methods.
  arXiv  Detail & Related papers  (2023-08-17T01:34:51Z)
• Quantization Aware Factorization for Deep Neural Network Compression [20.04951101799232]
  decomposition of convolutional and fully-connected layers is an effective way to reduce parameters and FLOP in neural networks. A conventional post-training quantization approach applied to networks with weights yields a drop in accuracy. This motivated us to develop an algorithm that finds decomposed approximation directly with quantized factors.
  arXiv  Detail & Related papers  (2023-08-08T21:38:02Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Understanding Weight Similarity of Neural Networks via Chain Normalization Rule and Hypothesis-Training-Testing [58.401504709365284]
  We present a weight similarity measure that can quantify the weight similarity of non-volution neural networks. We first normalize the weights of neural networks by a chain normalization rule, which is used to introduce weight-training representation learning. We extend traditional hypothesis-testing method to validate the hypothesis on the weight similarity of neural networks.
  arXiv  Detail & Related papers  (2022-08-08T19:11:03Z)
• A deep learning driven pseudospectral PCE based FFT homogenization algorithm for complex microstructures [68.8204255655161]
  It is shown that the proposed method is able to predict central moments of interest while being magnitudes faster to evaluate than traditional approaches. It is shown, that the proposed method is able to predict central moments of interest while being magnitudes faster to evaluate than traditional approaches.
  arXiv  Detail & Related papers  (2021-10-26T07:02:14Z)
• Non-intrusive reduced order modeling of poroelasticity of heterogeneous media based on a discontinuous Galerkin approximation [0.0]
  We present a non-intrusive model reduction framework for linear poroelasticity problems in heterogeneous porous media. We utilize the interior penalty discontinuous Galerkin (DG) method as a full order solver to handle discontinuity. We show that our framework provides reasonable approximations of the DG solution, but it is significantly faster.
  arXiv  Detail & Related papers  (2021-01-28T04:21:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.