Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practice

• URL: http://arxiv.org/abs/2405.11457v1
• Date: Sun, 19 May 2024 05:58:44 GMT
• Title: Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practice
• Authors: Yusheng Jiao, Feng Ling, Sina Heydari, Nicolas Heess, Josh Merel, Eva Kanso,
• Abstract summary: We present a concise exposition of the mathematical and algorithmic aspects of model-free reinforcement learning. We use textitactor-critic methods as a tool for investigating the feedback control underlying animal and robotic behavior.
• Score: 17.598549532513122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Animals and robots exist in a physical world and must coordinate their bodies to achieve behavioral objectives. With recent developments in deep reinforcement learning, it is now possible for scientists and engineers to obtain sensorimotor strategies (policies) for specific tasks using physically simulated bodies and environments. However, the utility of these methods goes beyond the constraints of a specific task; they offer an exciting framework for understanding the organization of an animal sensorimotor system in connection to its morphology and physical interaction with the environment, as well as for deriving general design rules for sensing and actuation in robotic systems. Algorithms and code implementing both learning agents and environments are increasingly available, but the basic assumptions and choices that go into the formulation of an embodied feedback control problem using deep reinforcement learning may not be immediately apparent. Here, we present a concise exposition of the mathematical and algorithmic aspects of model-free reinforcement learning, specifically through the use of \textit{actor-critic} methods, as a tool for investigating the feedback control underlying animal and robotic behavior.

Related papers

• Dynamic planning in hierarchical active inference [0.0]
  We refer to the ability of the human brain to infer and impose motor trajectories related to cognitive decisions. This study distances from traditional views centered on neural networks and reinforcement learning, and points toward a yet unexplored direction in active inference.
  arXiv  Detail & Related papers  (2024-02-18T17:32:53Z)
• Modular Neural Network Policies for Learning In-Flight Object Catching with a Robot Hand-Arm System [55.94648383147838]
  We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, and (iv) a grasping control policy trained to perform soft catching motions. We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight
  arXiv  Detail & Related papers  (2023-12-21T16:20:12Z)
• Adaptive User-centered Neuro-symbolic Learning for Multimodal Interaction with Autonomous Systems [0.0]
  Recent advances in machine learning have enabled autonomous systems to perceive and comprehend objects. It is essential to consider both the explicit teaching provided by humans and the implicit teaching obtained by observing human behavior. We argue for considering both types of inputs, as well as human-in-the-loop and incremental learning techniques.
  arXiv  Detail & Related papers  (2023-09-11T19:35:12Z)
• Incremental procedural and sensorimotor learning in cognitive humanoid robots [52.77024349608834]
  This work presents a cognitive agent that can learn procedures incrementally. We show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Results show that this approach is capable of solving complex tasks incrementally.
  arXiv  Detail & Related papers  (2023-04-30T22:51:31Z)
• Planning for Learning Object Properties [117.27898922118946]
  We formalize the problem of automatically training a neural network to recognize object properties as a symbolic planning problem. We use planning techniques to produce a strategy for automating the training dataset creation and the learning process. We provide an experimental evaluation in both a simulated and a real environment.
  arXiv  Detail & Related papers  (2023-01-15T09:37:55Z)
• Interpreting Neural Policies with Disentangled Tree Representations [58.769048492254555]
  We study interpretability of compact neural policies through the lens of disentangled representation. We leverage decision trees to obtain factors of variation for disentanglement in robot learning. We introduce interpretability metrics that measure disentanglement of learned neural dynamics.
  arXiv  Detail & Related papers  (2022-10-13T01:10:41Z)
• A neural net architecture based on principles of neural plasticity and development evolves to effectively catch prey in a simulated environment [2.834895018689047]
  A profound challenge for A-Life is to construct agents whose behavior is 'life-like' in a deep way. We propose an architecture and approach to constructing networks driving artificial agents, using processes analogous to the processes that construct and sculpt the brains of animals. We think this architecture may be useful for controlling small autonomous robots or drones, because it allows for a rapid response to changes in sensor inputs.
  arXiv  Detail & Related papers  (2022-01-28T05:10:56Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot [0.6193838300896449]
  We propose a decentralized organization as found in insect motor control for coordination of different legs. A concurrent local structure is able to learn better walking behavior.
  arXiv  Detail & Related papers  (2020-05-21T11:40:37Z)
• SAPIEN: A SimulAted Part-based Interactive ENvironment [77.4739790629284]
  SAPIEN is a realistic and physics-rich simulated environment that hosts a large-scale set for articulated objects. We evaluate state-of-the-art vision algorithms for part detection and motion attribute recognition as well as demonstrate robotic interaction tasks.
  arXiv  Detail & Related papers  (2020-03-19T00:11:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.