Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation

• URL: http://arxiv.org/abs/2406.06714v2
• Date: Mon, 07 Oct 2024 21:07:33 GMT
• Title: Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation
• Authors: Michelle Pan, Mariah Schrum, Vivek Myers, Erdem Bıyık, Anca Dragan,
• Abstract summary: Coprocessor Actor Critic is a novel, model-based reinforcement learning (MBRL) approach for learning neural coprocessor policies for brain stimulation. We show that our approach overcomes the limitations of traditional MFRL methods in terms of sample efficiency and task success.
• Score: 5.734832905595746
• License:
• Abstract: Adaptive brain stimulation can treat neurological conditions such as Parkinson's disease and post-stroke motor deficits by influencing abnormal neural activity. Because of patient heterogeneity, each patient requires a unique stimulation policy to achieve optimal neural responses. Model-free reinforcement learning (MFRL) holds promise in learning effective policies for a variety of similar control tasks, but is limited in domains like brain stimulation by a need for numerous costly environment interactions. In this work we introduce Coprocessor Actor Critic, a novel, model-based reinforcement learning (MBRL) approach for learning neural coprocessor policies for brain stimulation. Our key insight is that coprocessor policy learning is a combination of learning how to act optimally in the world and learning how to induce optimal actions in the world through stimulation of an injured brain. We show that our approach overcomes the limitations of traditional MFRL methods in terms of sample efficiency and task success and outperforms baseline MBRL approaches in a neurologically realistic model of an injured brain.

Related papers

• Brain-like Functional Organization within Large Language Models [58.93629121400745]
  The human brain has long inspired the pursuit of artificial intelligence (AI) Recent neuroimaging studies provide compelling evidence of alignment between the computational representation of artificial neural networks (ANNs) and the neural responses of the human brain to stimuli. In this study, we bridge this gap by directly coupling sub-groups of artificial neurons with functional brain networks (FBNs) This framework links the AN sub-groups to FBNs, enabling the delineation of brain-like functional organization within large language models (LLMs)
  arXiv  Detail & Related papers  (2024-10-25T13:15:17Z)
• Growing Deep Neural Network Considering with Similarity between Neurons [4.32776344138537]
  We explore a novel approach of progressively increasing neuron numbers in compact models during training phases. We propose a method that reduces feature extraction biases and neuronal redundancy by introducing constraints based on neuron similarity distributions. Results on CIFAR-10 and CIFAR-100 datasets demonstrated accuracy improvement.
  arXiv  Detail & Related papers  (2024-08-23T11:16:37Z)
• Lifelong Reinforcement Learning via Neuromodulation [13.765526492965853]
  Evolution has imbued animals and humans with highly effective adaptive learning functions and decision-making strategies. Central to these theories and integrating evidence from neuroscience into learning is the neuromodulatory system.
  arXiv  Detail & Related papers  (2024-08-15T22:53:35Z)
• Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling [52.06722364186432]
  We propose a biologically-informed framework for enhancing artificial neural networks (ANNs) Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors. We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
  arXiv  Detail & Related papers  (2024-07-05T14:11:28Z)
• A Differentiable Approach to Multi-scale Brain Modeling [3.5874544981360987]
  We present a multi-scale differentiable brain modeling workflow utilizing BrainPy, a unique differentiable brain simulator. At the single-neuron level, we implement differentiable neuron models and employ gradient methods to optimize their fit to electrophysiological data. On the network level, we incorporate connectomic data to construct biologically constrained network models.
  arXiv  Detail & Related papers  (2024-06-28T07:41:31Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Control of synaptic plasticity via the fusion of reinforcement learning and unsupervised learning in neural networks [0.0]
  In cognitive neuroscience, it is widely accepted that synaptic plasticity plays an essential role in our amazing learning capability. With this inspiration, a new learning rule is proposed via the fusion of reinforcement learning and unsupervised learning. In the proposed computational model, the nonlinear optimal control theory is used to resemble the error feedback loop systems.
  arXiv  Detail & Related papers  (2023-03-26T12:18:03Z)
• Neural Co-Processors for Restoring Brain Function: Results from a Cortical Model of Grasping [0.0]
  We propose "neural co-processors" which use artificial neural networks and deep learning to learn optimal closed-loop stimulation policies. The co-processor adapts the stimulation policy as the biological circuit itself adapts to the stimulation, achieving a form of brain-device co-adaptation.
  arXiv  Detail & Related papers  (2022-10-19T04:13:33Z)
• Multimodal foundation models are better simulators of the human brain [65.10501322822881]
  We present a newly-designed multimodal foundation model pre-trained on 15 million image-text pairs. We find that both visual and lingual encoders trained multimodally are more brain-like compared with unimodal ones.
  arXiv  Detail & Related papers  (2022-08-17T12:36:26Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.