Indeterminate Probability Neural Network

• URL: http://arxiv.org/abs/2303.11536v1
• Date: Tue, 21 Mar 2023 01:57:40 GMT
• Title: Indeterminate Probability Neural Network
• Authors: Tao Yang, Chuang Liu, Xiaofeng Ma, Weijia Lu, Ning Wu, Bingyang Li, Zhifei Yang, Peng Liu, Lin Sun, Xiaodong Zhang, Can Zhang
• Abstract summary: In this paper, we propose a new general probability theory, which is an extension of classical probability theory. For our proposed neural network framework, the output of neural network is defined as probability events. IPNN is capable of making very large classification with very small neural network, e.g. model with 100 output nodes can classify 10 billion categories.
• Score: 20.993728880886994
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose a new general model called IPNN - Indeterminate Probability Neural Network, which combines neural network and probability theory together. In the classical probability theory, the calculation of probability is based on the occurrence of events, which is hardly used in current neural networks. In this paper, we propose a new general probability theory, which is an extension of classical probability theory, and makes classical probability theory a special case to our theory. Besides, for our proposed neural network framework, the output of neural network is defined as probability events, and based on the statistical analysis of these events, the inference model for classification task is deduced. IPNN shows new property: It can perform unsupervised clustering while doing classification. Besides, IPNN is capable of making very large classification with very small neural network, e.g. model with 100 output nodes can classify 10 billion categories. Theoretical advantages are reflected in experimental results.

Related papers

• Utility-Probability Duality of Neural Networks [4.871730595406078]
  We propose an alternative utility-based explanation to the standard supervised learning procedure in deep learning. The basic idea is to interpret the learned neural network not as a probability model but as an ordinal utility function. We show that for all neural networks with softmax outputs, the SGD learning dynamic of maximum likelihood estimation can be seen as an iteration process.
  arXiv  Detail & Related papers  (2023-05-24T08:09:07Z)
• Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Network [30.924449325020767]
  Lottery Ticket Hypothesis (LTH) states that a randomly-d large neural network contains a small sub-network. LTH opens up a new path for pruning network.
  arXiv  Detail & Related papers  (2023-05-20T09:27:34Z)
• Continuous Indeterminate Probability Neural Network [4.198538504785438]
  This paper introduces a general model called CIPNN - Continuous Indeterminate Probability Neural Network. CIPNN is based on IPNN, which is used for discrete latent random variables. We propose a new method to visualize the latent random variables, we use one of N dimensional latent variables as a decoder.
  arXiv  Detail & Related papers  (2023-03-23T00:11:17Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks [91.3755431537592]
  We study how random pruning of the weights affects a neural network's neural kernel (NTK) In particular, this work establishes an equivalence of the NTKs between a fully-connected neural network and its randomly pruned version.
  arXiv  Detail & Related papers  (2022-03-27T15:22:19Z)
• On some theoretical limitations of Generative Adversarial Networks [77.34726150561087]
  It is a general assumption that GANs can generate any probability distribution. We provide a new result based on Extreme Value Theory showing that GANs can't generate heavy tailed distributions.
  arXiv  Detail & Related papers  (2021-10-21T06:10:38Z)
• Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks [79.74580058178594]
  We analyze the performance of training a pruned neural network by analyzing the geometric structure of the objective function. We show that the convex region near a desirable model with guaranteed generalization enlarges as the neural network model is pruned.
  arXiv  Detail & Related papers  (2021-10-12T01:11:07Z)
• The Separation Capacity of Random Neural Networks [78.25060223808936]
  We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability. We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
  arXiv  Detail & Related papers  (2021-07-31T10:25:26Z)
• Mitigating Performance Saturation in Neural Marked Point Processes: Architectures and Loss Functions [50.674773358075015]
  We propose a simple graph-based network structure called GCHP, which utilizes only graph convolutional layers. We show that GCHP can significantly reduce training time and the likelihood ratio loss with interarrival time probability assumptions can greatly improve the model performance.
  arXiv  Detail & Related papers  (2021-07-07T16:59:14Z)
• Probabilistic Deep Learning with Probabilistic Neural Networks and Deep Probabilistic Models [0.6091702876917281]
  We distinguish two approaches to probabilistic deep learning: probabilistic neural networks and deep probabilistic models. Probabilistic deep learning is deep learning that accounts for uncertainty, both model uncertainty and data uncertainty.
  arXiv  Detail & Related papers  (2021-05-31T22:13:21Z)
• Perceptron Theory Can Predict the Accuracy of Neural Networks [6.136302173351179]
  Multilayer neural networks set the current state of the art for many technical classification problems. But, these networks are still, essentially, black boxes in terms of analyzing them and predicting their performance. Here, we develop a statistical theory for the one-layer perceptron and show that it can predict performances of a surprisingly large variety of neural networks.
  arXiv  Detail & Related papers  (2020-12-14T19:02:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.