Dynamic Bayesian Neural Networks

• URL: http://arxiv.org/abs/2004.06963v2
• Date: Wed, 24 Jun 2020 14:29:17 GMT
• Title: Dynamic Bayesian Neural Networks
• Authors: Lorenzo Rimella and Nick Whiteley
• Abstract summary: We define an evolving in time neural network called a Hidden Markov neural network. Weights of a feed-forward neural network are modelled with the hidden states of a Hidden Markov model. A filtering algorithm is used to learn a variational approximation to the evolving in time posterior over the weights.
• Score: 2.28438857884398
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We define an evolving in time Bayesian neural network called a Hidden Markov neural network. The weights of a feed-forward neural network are modelled with the hidden states of a Hidden Markov model, whose observed process is given by the available data. A filtering algorithm is used to learn a variational approximation to the evolving in time posterior over the weights. Training is pursued through a sequential version of Bayes by Backprop Blundell et al. 2015, which is enriched with a stronger regularization technique called variational DropConnect. The experiments test variational DropConnect on MNIST and display the performance of Hidden Markov neural networks on time series.

Related papers

• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Generative Kaleidoscopic Networks [2.321684718906739]
  We utilize this property of neural networks to design a dataset kaleidoscope, termed as Generative Kaleidoscopic Networks' We observed this phenomenon to various degrees for the other deep learning architectures like CNNs, Transformers & U-Nets.
  arXiv  Detail & Related papers  (2024-02-19T02:48:40Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Spiking neural networks with Hebbian plasticity for unsupervised representation learning [0.0]
  We introduce a novel spiking neural network model for learning distributed internal representations from data in an unsupervised procedure. We employ an online correlation-based Hebbian-Bayesian learning and rewiring mechanism, shown previously to perform representation learning, into a spiking neural network. We show performance close to the non-spiking BCPNN, and competitive with other Hebbian-based spiking networks when trained on MNIST and F-MNIST machine learning benchmarks.
  arXiv  Detail & Related papers  (2023-05-05T22:34:54Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics [85.31710759801705]
  Current practice requires expensive computational costs in model training for performance prediction. We propose a novel framework for neural network selection by analyzing the governing dynamics over synaptic connections (edges) during training. Our framework is built on the fact that back-propagation during neural network training is equivalent to the dynamical evolution of synaptic connections.
  arXiv  Detail & Related papers  (2022-01-11T20:53:15Z)
• Stochastic Recurrent Neural Network for Multistep Time Series Forecasting [0.0]
  We leverage advances in deep generative models and the concept of state space models to propose an adaptation of the recurrent neural network for time series forecasting. Our model preserves the architectural workings of a recurrent neural network for which all relevant information is encapsulated in its hidden states, and this flexibility allows our model to be easily integrated into any deep architecture for sequential modelling.
  arXiv  Detail & Related papers  (2021-04-26T01:43:43Z)
• Modeling the Nonsmoothness of Modern Neural Networks [35.93486244163653]
  We quantify the nonsmoothness using a feature named the sum of the magnitude of peaks (SMP) We envision that the nonsmoothness feature can potentially be used as a forensic tool for regression-based applications of neural networks.
  arXiv  Detail & Related papers  (2021-03-26T20:55:19Z)
• A Bayesian Perspective on Training Speed and Model Selection [51.15664724311443]
  We show that a measure of a model's training speed can be used to estimate its marginal likelihood. We verify our results in model selection tasks for linear models and for the infinite-width limit of deep neural networks. Our results suggest a promising new direction towards explaining why neural networks trained with gradient descent are biased towards functions that generalize well.
  arXiv  Detail & Related papers  (2020-10-27T17:56:14Z)
• Stochastic Bayesian Neural Networks [0.0]
  We build on variational inference techniques for bayesian neural networks using the original Evidence Lower Bound. We present a bayesian neural network in which we maximize Evidence Lower Bound using a new objective function which we name as Evidence Lower Bound.
  arXiv  Detail & Related papers  (2020-08-12T19:48:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.