Interpretable Neural Networks with Random Constructive Algorithm

• URL: http://arxiv.org/abs/2307.00185v3
• Date: Sun, 14 Apr 2024 13:06:24 GMT
• Title: Interpretable Neural Networks with Random Constructive Algorithm
• Authors: Jing Nan, Wei Dai,
• Abstract summary: This paper introduces an Interpretable Neural Network (INN) incorporating spatial information to tackle the opaque parameterization process of random weighted neural networks. It devises a geometric relationship strategy using a pool of candidate nodes and established relationships to select node parameters conducive to network convergence.
• Score: 3.1200894334384954
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces an Interpretable Neural Network (INN) incorporating spatial information to tackle the opaque parameterization process of random weighted neural networks. The INN leverages spatial information to elucidate the connection between parameters and network residuals. Furthermore, it devises a geometric relationship strategy using a pool of candidate nodes and established relationships to select node parameters conducive to network convergence. Additionally, a lightweight version of INN tailored for large-scale data modeling tasks is proposed. The paper also showcases the infinite approximation property of INN. Experimental findings on various benchmark datasets and real-world industrial cases demonstrate INN's superiority over other neural networks of the same type in terms of modeling speed, accuracy, and network structure.

Related papers

• Steinmetz Neural Networks for Complex-Valued Data [23.80312814400945]
  We introduce a new approach to processing complex-valued data using DNNs consisting of parallel real-valuedetzworks with coupled outputs. Our proposed class of architectures, referred to as Steinmetz Neural Networks, leverage multi-view learning to construct more interpretable representations within the latent space. Our numerical experiments depict the improved performance and to additive noise, afforded by these networks on benchmark datasets and synthetic examples.
  arXiv  Detail & Related papers  (2024-09-16T08:26:06Z)
• Unveiling the Power of Sparse Neural Networks for Feature Selection [60.50319755984697]
  Sparse Neural Networks (SNNs) have emerged as powerful tools for efficient feature selection. We show that SNNs trained with dynamic sparse training (DST) algorithms can achieve, on average, more than $50%$ memory and $55%$ FLOPs reduction. Our findings show that feature selection with SNNs trained with DST algorithms can achieve, on average, more than $50%$ memory and $55%$ FLOPs reduction.
  arXiv  Detail & Related papers  (2024-08-08T16:48:33Z)
• Set-based Neural Network Encoding Without Weight Tying [91.37161634310819]
  We propose a neural network weight encoding method for network property prediction. Our approach is capable of encoding neural networks in a model zoo of mixed architecture. We introduce two new tasks for neural network property prediction: cross-dataset and cross-architecture.
  arXiv  Detail & Related papers  (2023-05-26T04:34:28Z)
• Deep Neural Networks as Complex Networks [1.704936863091649]
  We use Complex Network Theory to represent Deep Neural Networks (DNNs) as directed weighted graphs. We introduce metrics to study DNNs as dynamical systems, with a granularity that spans from weights to layers, including neurons. We show that our metrics discriminate low vs. high performing networks.
  arXiv  Detail & Related papers  (2022-09-12T16:26:04Z)
• Linear Leaky-Integrate-and-Fire Neuron Model Based Spiking Neural Networks and Its Mapping Relationship to Deep Neural Networks [7.840247953745616]
  Spiking neural networks (SNNs) are brain-inspired machine learning algorithms with merits such as biological plausibility and unsupervised learning capability. This paper establishes a precise mathematical mapping between the biological parameters of the Linear Leaky-Integrate-and-Fire model (LIF)/SNNs and the parameters of ReLU-AN/Deep Neural Networks (DNNs)
  arXiv  Detail & Related papers  (2022-05-31T17:02:26Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Deep Neural Networks and PIDE discretizations [2.4063592468412276]
  We propose neural networks that tackle the problems of stability and field-of-view of a Convolutional Neural Network (CNN) We propose integral-based spatially nonlocal operators which are related to global weighted Laplacian, fractional Laplacian and fractional inverse Laplacian operators. We test the effectiveness of the proposed neural architectures on benchmark image classification datasets and semantic segmentation tasks in autonomous driving.
  arXiv  Detail & Related papers  (2021-08-05T08:03:01Z)
• Learning Autonomy in Management of Wireless Random Networks [102.02142856863563]
  This paper presents a machine learning strategy that tackles a distributed optimization task in a wireless network with an arbitrary number of randomly interconnected nodes. We develop a flexible deep neural network formalism termed distributed message-passing neural network (DMPNN) with forward and backward computations independent of the network topology.
  arXiv  Detail & Related papers  (2021-06-15T09:03:28Z)
• Explore the Knowledge contained in Network Weights to Obtain Sparse Neural Networks [2.649890751459017]
  This paper proposes a novel learning approach to obtain sparse fully connected layers in neural networks (NNs) automatically. We design a switcher neural network (SNN) to optimize the structure of the task neural network (TNN)
  arXiv  Detail & Related papers  (2021-03-26T11:29:40Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.