Understanding Generalization, Robustness, and Interpretability in Low-Capacity Neural Networks

• URL: http://arxiv.org/abs/2507.16278v1
• Date: Tue, 22 Jul 2025 06:43:03 GMT
• Title: Understanding Generalization, Robustness, and Interpretability in Low-Capacity Neural Networks
• Authors: Yash Kumar,
• Abstract summary: We introduce a framework to investigate capacity, sparsity, and robustness in low-capacity networks.<n>We show that trained networks are robust to extreme magnitude pruning (up to 95% sparsity)<n>This work provides a clear, empirical demonstration of the trade-offs governing simple neural networks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although modern deep learning often relies on massive over-parameterized models, the fundamental interplay between capacity, sparsity, and robustness in low-capacity networks remains a vital area of study. We introduce a controlled framework to investigate these properties by creating a suite of binary classification tasks from the MNIST dataset with increasing visual difficulty (e.g., 0 and 1 vs. 4 and 9). Our experiments reveal three core findings. First, the minimum model capacity required for successful generalization scales directly with task complexity. Second, these trained networks are robust to extreme magnitude pruning (up to 95% sparsity), revealing the existence of sparse, high-performing subnetworks. Third, we show that over-parameterization provides a significant advantage in robustness against input corruption. Interpretability analysis via saliency maps further confirms that these identified sparse subnetworks preserve the core reasoning process of the original dense models. This work provides a clear, empirical demonstration of the foundational trade-offs governing simple neural networks.

Related papers

• Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks [13.983863226803336]
  We show that neural networks trained by gradient descent tend to rely on an average of features for classification.<n>We prove that gradient descent biases the network towards feature averaging, where the weights of each hidden neuron represent an average of the cluster centers.<n>We prove that a two-layer ReLU network can achieve optimal robustness when trained to classify individual features.
  arXiv  Detail & Related papers  (2024-10-14T09:28:32Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [64.04617968947697]
  We introduce a novel data-model co-design perspective: to promote superior weight sparsity. Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
  arXiv  Detail & Related papers  (2023-12-03T13:50:24Z)
• Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks [44.31729147722701]
  We show that three-layer neural networks have provably richer feature learning capabilities than two-layer networks.<n>This work makes progress towards understanding the provable benefit of three-layer neural networks over two-layer networks in the feature learning regime.
  arXiv  Detail & Related papers  (2023-05-11T17:19:30Z)
• Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks [89.28881869440433]
  This paper provides the first theoretical characterization of joint edge-model sparse learning for graph neural networks (GNNs) It proves analytically that both sampling important nodes and pruning neurons with the lowest-magnitude can reduce the sample complexity and improve convergence without compromising the test accuracy.
  arXiv  Detail & Related papers  (2023-02-06T16:54:20Z)
• Overcoming Simplicity Bias in Deep Networks using a Feature Sieve [5.33024001730262]
  We propose a direct, interventional method for addressing simplicity bias in deep networks. We aim to automatically identify and suppress easily-computable spurious features in lower layers of the network. We report substantial gains on many real-world debiasing benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T21:11:13Z)
• Neural networks trained with SGD learn distributions of increasing complexity [78.30235086565388]
  We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics. We then exploit higher-order statistics only later during training. We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
  arXiv  Detail & Related papers  (2022-11-21T15:27:22Z)
• Functional Network: A Novel Framework for Interpretability of Deep Neural Networks [2.641939670320645]
  We propose a novel framework for interpretability of deep neural networks, that is, the functional network. In our experiments, the mechanisms of regularization methods, namely, batch normalization and dropout, are revealed.
  arXiv  Detail & Related papers  (2022-05-24T01:17:36Z)
• Understanding Adversarial Robustness from Feature Maps of Convolutional Layers [23.42376264664302]
  Anti-perturbation ability of a neural network mainly relies on two factors: model capacity and anti-perturbation ability. We study the anti-perturbation ability of the network from the feature maps of convolutional layers. Non-trivial improvements in terms of both natural accuracy and adversarial robustness can be achieved under various attack and defense mechanisms.
  arXiv  Detail & Related papers  (2022-02-25T00:14:59Z)
• Analytic Insights into Structure and Rank of Neural Network Hessian Maps [32.90143789616052]
  Hessian of a neural network captures parameter interactions through second-order derivatives of the loss. We develop theoretical tools to analyze the range of the Hessian map, providing us with a precise understanding of its rank deficiency. This yields exact formulas and tight upper bounds for the Hessian rank of deep linear networks.
  arXiv  Detail & Related papers  (2021-06-30T17:29:58Z)
• Non-Singular Adversarial Robustness of Neural Networks [58.731070632586594]
  Adrial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. We formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.
  arXiv  Detail & Related papers  (2021-02-23T20:59:30Z)
• The Self-Simplifying Machine: Exploiting the Structure of Piecewise Linear Neural Networks to Create Interpretable Models [0.0]
  We introduce novel methodology toward simplification and increased interpretability of Piecewise Linear Neural Networks for classification tasks. Our methods include the use of a trained, deep network to produce a well-performing, single-hidden-layer network without further training. On these methods, we conduct preliminary studies of model performance, as well as a case study on Wells Fargo's Home Lending dataset.
  arXiv  Detail & Related papers  (2020-12-02T16:02:14Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.