Continuous Indeterminate Probability Neural Network

• URL: http://arxiv.org/abs/2303.12964v1
• Date: Thu, 23 Mar 2023 00:11:17 GMT
• Title: Continuous Indeterminate Probability Neural Network
• Authors: Tao Yang
• Abstract summary: This paper introduces a general model called CIPNN - Continuous Indeterminate Probability Neural Network. CIPNN is based on IPNN, which is used for discrete latent random variables. We propose a new method to visualize the latent random variables, we use one of N dimensional latent variables as a decoder.
• Score: 4.198538504785438
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper introduces a general model called CIPNN - Continuous Indeterminate Probability Neural Network, and this model is based on IPNN, which is used for discrete latent random variables. Currently, posterior of continuous latent variables is regarded as intractable, with the new theory proposed by IPNN this problem can be solved. Our contributions are Four-fold. First, we derive the analytical solution of the posterior calculation of continuous latent random variables and propose a general classification model (CIPNN). Second, we propose a general auto-encoder called CIPAE - Continuous Indeterminate Probability Auto-Encoder, the decoder part is not a neural network and uses a fully probabilistic inference model for the first time. Third, we propose a new method to visualize the latent random variables, we use one of N dimensional latent variables as a decoder to reconstruct the input image, which can work even for classification tasks, in this way, we can see what each latent variable has learned. Fourth, IPNN has shown great classification capability, CIPNN has pushed this classification capability to infinity. Theoretical advantages are reflected in experimental results.

Related papers

• Continual learning with the neural tangent ensemble [0.6137178191238463]
  We show that a neural network with N parameters can be interpreted as a weighted ensemble of N classifiers. We derive the likelihood and posterior probability of each expert given past data. Surprisingly, we learn that the posterior updates for these experts are equivalent to a scaled and projected form of gradient descent.
  arXiv  Detail & Related papers  (2024-08-30T16:29:09Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• An interpretable neural network-based non-proportional odds model for ordinal regression [3.0277213703725767]
  This study proposes an interpretable neural network-based non-proportional odds model (N$3$POM) for ordinal regression. N$3$POM is different from conventional approaches to ordinal regression with non-proportional models in several ways.
  arXiv  Detail & Related papers  (2023-03-31T06:40:27Z)
• Indeterminate Probability Neural Network [20.993728880886994]
  In this paper, we propose a new general probability theory, which is an extension of classical probability theory. For our proposed neural network framework, the output of neural network is defined as probability events. IPNN is capable of making very large classification with very small neural network, e.g. model with 100 output nodes can classify 10 billion categories.
  arXiv  Detail & Related papers  (2023-03-21T01:57:40Z)
• Censored Quantile Regression Neural Networks [24.118509578363593]
  This paper considers doing quantile regression on censored data using neural networks (NNs) We show how an algorithm popular in linear models can be applied to NNs. Our major contribution is a novel algorithm that simultaneously optimises a grid of quantiles output by a single NN.
  arXiv  Detail & Related papers  (2022-05-26T17:10:28Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• Rethinking Nearest Neighbors for Visual Classification [56.00783095670361]
  k-NN is a lazy learning method that aggregates the distance between the test image and top-k neighbors in a training set. We adopt k-NN with pre-trained visual representations produced by either supervised or self-supervised methods in two steps. Via extensive experiments on a wide range of classification tasks, our study reveals the generality and flexibility of k-NN integration.
  arXiv  Detail & Related papers  (2021-12-15T20:15:01Z)
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking [63.49779304362376]
  Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models. We introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges. We show that we can drop a large proportion of edges without deteriorating the performance of the model.
  arXiv  Detail & Related papers  (2020-10-01T17:51:19Z)
• Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables [37.43541345780632]
  We present a new variant of Neural Process (NP) model that we call Doubly Variational Neural Process (DSVNP) This model combines the global latent variable and local latent variables for prediction. We evaluate this model in several experiments, and our results demonstrate competitive prediction performance in multi-output regression and uncertainty estimation in classification.
  arXiv  Detail & Related papers  (2020-08-21T13:32:12Z)
• Improving predictions of Bayesian neural nets via local linearization [79.21517734364093]
  We argue that the Gauss-Newton approximation should be understood as a local linearization of the underlying Bayesian neural network (BNN) Because we use this linearized model for posterior inference, we should also predict using this modified model instead of the original one. We refer to this modified predictive as "GLM predictive" and show that it effectively resolves common underfitting problems of the Laplace approximation.
  arXiv  Detail & Related papers  (2020-08-19T12:35:55Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.