Deep inference of latent dynamics with spatio-temporal super-resolution using selective backpropagation through time

• URL: http://arxiv.org/abs/2111.00070v1
• Date: Fri, 29 Oct 2021 20:18:29 GMT
• Title: Deep inference of latent dynamics with spatio-temporal super-resolution using selective backpropagation through time
• Authors: Feng Zhu, Andrew R. Sedler, Harrison A. Grier, Nauman Ahad, Mark A. Davenport, Matthew T. Kaufman, Andrea Giovannucci, Chethan Pandarinath
• Abstract summary: Modern neural interfaces allow access to the activity of up to a million neurons within brain circuits. bandwidth limits often create a trade-off between greater spatial sampling (more channels or pixels) and frequency of temporal sampling. Here we demonstrate that it is possible to obtain super-resolution in neuronal time series by exploiting relationships among neurons.
• Score: 15.648009434801885
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Modern neural interfaces allow access to the activity of up to a million neurons within brain circuits. However, bandwidth limits often create a trade-off between greater spatial sampling (more channels or pixels) and the temporal frequency of sampling. Here we demonstrate that it is possible to obtain spatio-temporal super-resolution in neuronal time series by exploiting relationships among neurons, embedded in latent low-dimensional population dynamics. Our novel neural network training strategy, selective backpropagation through time (SBTT), enables learning of deep generative models of latent dynamics from data in which the set of observed variables changes at each time step. The resulting models are able to infer activity for missing samples by combining observations with learned latent dynamics. We test SBTT applied to sequential autoencoders and demonstrate more efficient and higher-fidelity characterization of neural population dynamics in electrophysiological and calcium imaging data. In electrophysiology, SBTT enables accurate inference of neuronal population dynamics with lower interface bandwidths, providing an avenue to significant power savings for implanted neuroelectronic interfaces. In applications to two-photon calcium imaging, SBTT accurately uncovers high-frequency temporal structure underlying neural population activity, substantially outperforming the current state-of-the-art. Finally, we demonstrate that performance could be further improved by using limited, high-bandwidth sampling to pretrain dynamics models, and then using SBTT to adapt these models for sparsely-sampled data.

Related papers

• Zero-Shot Temporal Resolution Domain Adaptation for Spiking Neural Networks [3.2366933261812076]
  Spiking Neural Networks (SNNs) are biologically-inspired deep neural networks that efficiently extract temporal information. SNN model parameters are sensitive to temporal resolution, leading to significant performance drops when the temporal resolution of target data at the edge is not the same. We propose three novel domain adaptation methods for adapting neuron parameters to account for the change in time resolution without re-training on target time-resolution.
  arXiv  Detail & Related papers  (2024-11-07T14:58:51Z)
• Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks [50.32980443749865]
  Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biologicalability. Current SNNs struggle to balance accuracy and latency in neuromorphic datasets. We propose Step-wise Distillation (HSD) method, tailored for neuromorphic datasets.
  arXiv  Detail & Related papers  (2024-09-19T06:52:34Z)
• A frugal Spiking Neural Network for unsupervised classification of continuous multivariate temporal data [0.0]
  Spiking Neural Networks (SNNs) are neuromorphic and use more biologically plausible neurons with evolving membrane potentials. We introduce here a frugal single-layer SNN designed for fully unsupervised identification and classification of multivariate temporal patterns in continuous data.
  arXiv  Detail & Related papers  (2024-08-08T08:15:51Z)
• Long Short-term Memory with Two-Compartment Spiking Neuron [64.02161577259426]
  We propose a novel biologically inspired Long Short-Term Memory Leaky Integrate-and-Fire spiking neuron model, dubbed LSTM-LIF. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, strong network generalizability, and high energy efficiency of the proposed LSTM-LIF model. This work, therefore, opens up a myriad of opportunities for resolving challenging temporal processing tasks on emerging neuromorphic computing machines.
  arXiv  Detail & Related papers  (2023-07-14T08:51:03Z)
• The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks [64.08042492426992]
  We introduce the Expressive Memory (ELM) neuron model, a biologically inspired model of a cortical neuron. Our ELM neuron can accurately match the aforementioned input-output relationship with under ten thousand trainable parameters. We evaluate it on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets.
  arXiv  Detail & Related papers  (2023-06-14T13:34:13Z)
• Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unit [56.801856519460465]
  Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations. We demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting.
  arXiv  Detail & Related papers  (2023-04-14T09:39:06Z)
• STNDT: Modeling Neural Population Activity with a Spatiotemporal Transformer [19.329190789275565]
  We introduce SpatioTemporal Neural Data Transformer (STNDT), an NDT-based architecture that explicitly models responses of individual neurons. We show that our model achieves state-of-the-art performance on ensemble level in estimating neural activities across four neural datasets.
  arXiv  Detail & Related papers  (2022-06-09T18:54:23Z)
• An advanced spatio-temporal convolutional recurrent neural network for storm surge predictions [73.4962254843935]
  We study the capability of artificial neural network models to emulate storm surge based on the storm track/size/intensity history. This study presents a neural network model that can predict storm surge, informed by a database of synthetic storm simulations.
  arXiv  Detail & Related papers  (2022-04-18T23:42:18Z)
• Ultra-low Latency Spiking Neural Networks with Spatio-Temporal Compression and Synaptic Convolutional Block [4.081968050250324]
  Spiking neural networks (SNNs) have neuro-temporal information capability, low processing feature, and high biological plausibility. Neuro-MNIST, CIFAR10-S, DVS128 gesture datasets need to aggregate individual events into frames with a higher temporal resolution for event stream classification. We propose a processing-temporal compression method to aggregate individual events into a few time steps of NIST current to reduce the training and inference latency.
  arXiv  Detail & Related papers  (2022-03-18T15:14:13Z)
• Representation learning for neural population activity with Neural Data Transformers [3.4376560669160394]
  We introduce the Neural Data Transformer (NDT), a non-recurrent alternative to explicit dynamics models. NDT enables 3.9ms inference, well within the loop time of real-time applications and more than 6 times faster than recurrent baselines.
  arXiv  Detail & Related papers  (2021-08-02T23:36:39Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.