Novel Reinforcement Learning Algorithm for Suppressing Synchronization in Closed Loop Deep Brain Stimulators

• URL: http://arxiv.org/abs/2212.13260v1
• Date: Sun, 25 Dec 2022 11:29:55 GMT
• Title: Novel Reinforcement Learning Algorithm for Suppressing Synchronization in Closed Loop Deep Brain Stimulators
• Authors: Harsh Agarwal, Heena Rathore
• Abstract summary: Parkinson's disease is marked by altered and increased firing characteristics of pathological oscillations in the brain. Deep brain stimulators (DBS) are used to examine and regulate the synchronization and pathological oscillations in motor circuits. This research proposes a novel reinforcement learning framework for suppressing the synchronization in neuronal activity during episodes of neurological disorders with less power consumption.
• Score: 0.6294759639481188
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Parkinson's disease is marked by altered and increased firing characteristics of pathological oscillations in the brain. In other words, it causes abnormal synchronous oscillations and suppression during neurological processing. In order to examine and regulate the synchronization and pathological oscillations in motor circuits, deep brain stimulators (DBS) are used. Although machine learning methods have been applied for the investigation of suppression, these models require large amounts of training data and computational power, both of which pose challenges to resource-constrained DBS. This research proposes a novel reinforcement learning (RL) framework for suppressing the synchronization in neuronal activity during episodes of neurological disorders with less power consumption. The proposed RL algorithm comprises an ensemble of a temporal representation of stimuli and a twin-delayed deep deterministic (TD3) policy gradient algorithm. We quantify the stability of the proposed framework to noise and reduced synchrony using RL for three pathological signaling regimes: regular, chaotic, and bursting, and further eliminate the undesirable oscillations. Furthermore, metrics such as evaluation rewards, energy supplied to the ensemble, and the mean point of convergence were used and compared to other RL algorithms, specifically the Advantage actor critic (A2C), the Actor critic with Kronecker-featured trust region (ACKTR), and the Proximal policy optimization (PPO).

Related papers

• REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates [54.96885726053036]
  This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks.
  arXiv  Detail & Related papers  (2024-06-03T16:30:19Z)
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
  We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks. We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network. Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
  arXiv  Detail & Related papers  (2024-04-22T09:40:07Z)
• {\epsilon}-Neural Thompson Sampling of Deep Brain Stimulation for Parkinson Disease Treatment [15.303196613362099]
  We propose a contextual multi-armed bandits (CMAB) solution for a Deep Brain Stimulation (DBS) device. We define the context as the signals capturing irregular neuronal firing activities in the basal ganglia (BG) regions. An epsilon-exploring strategy is introduced on top of the classic Thompson sampling method, leading to an algorithm called epsilon-NeuralTS.
  arXiv  Detail & Related papers  (2024-03-11T15:33:40Z)
• Learning Control Policies of Hodgkin-Huxley Neuronal Dynamics [1.629803445577911]
  We approximate the value function offline using a neural network to enable generating controls (stimuli) in real time via the feedback form. Our numerical experiments illustrate the accuracy of our approach for out-of-distribution samples and the robustness to moderate shocks and disturbances in the system.
  arXiv  Detail & Related papers  (2023-11-13T18:53:50Z)
• Autonomous coherence protection of a two-level system in a fluctuating environment [68.8204255655161]
  We re-examine a scheme originally intended to remove the effects of static Doppler broadening from an ensemble of non-interacting two-level systems (qubits) We demonstrate that this scheme is far more powerful and can also protect a single (or even an ensemble) qubit's energy levels from noise which depends on both time and space.
  arXiv  Detail & Related papers  (2023-02-08T01:44:30Z)
• Limits of Entrainment of Circadian Neuronal Networks [0.0]
  Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals. We study a modern computational neuroscience model to determine the limits of circadian synchronization to external light signals of different frequency and duty cycle.
  arXiv  Detail & Related papers  (2022-08-23T17:57:21Z)
• Neural optimal feedback control with local learning rules [67.5926699124528]
  A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. We introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach.
  arXiv  Detail & Related papers  (2021-11-12T20:02:00Z)
• Spatiotemporal Spike-Pattern Selectivity in Single Mixed-Signal Neurons with Balanced Synapses [0.27998963147546135]
  Mixed-signal neuromorphic processors could be used for inference and learning. We show how inhomogeneous synaptic circuits could be utilized for resource-efficient implementation of network layers.
  arXiv  Detail & Related papers  (2021-06-10T12:04:03Z)
• Phase Retrieval using Expectation Consistent Signal Recovery Algorithm based on Hypernetwork [73.94896986868146]
  Phase retrieval is an important component in modern computational imaging systems. Recent advances in deep learning have opened up a new possibility for robust and fast PR. We develop a novel framework for deep unfolding to overcome the existing limitations.
  arXiv  Detail & Related papers  (2021-01-12T08:36:23Z)
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso [102.84915019938413]
  Non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG) offer promise of non-invasive techniques. The problem of source localization, or source imaging, poses however a high-dimensional statistical inference challenge. We propose an ensemble of desparsified multi-task Lasso (ecd-MTLasso) to deal with this problem.
  arXiv  Detail & Related papers  (2020-09-29T21:17:16Z)
• Reinforcement Learning Framework for Deep Brain Stimulation Study [10.505656411009388]
  Malfunctioning neurons in the brain sometimes operate synchronously, reportedly causing many neurological diseases. We present the first Reinforcement Learning gym framework that emulates this collective behavior of neurons. We successfully suppress synchrony via RL for three pathological signaling regimes, characterize the framework's stability to noise, and further remove the unwanted oscillations by engaging multiple PPO agents.
  arXiv  Detail & Related papers  (2020-02-22T16:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.