Temporally Layered Architecture for Adaptive, Distributed and Continuous
Control
- URL: http://arxiv.org/abs/2301.00723v1
- Date: Sun, 25 Dec 2022 08:46:22 GMT
- Title: Temporally Layered Architecture for Adaptive, Distributed and Continuous
Control
- Authors: Devdhar Patel, Joshua Russell, Francesca Walsh, Tauhidur Rahman,
Terrance Sejnowski, Hava Siegelmann
- Abstract summary: We present temporally layered architecture (TLA), a biologically inspired system for temporally adaptive distributed control.
TLA layers a fast and a slow controller together to achieve temporal abstraction that allows each layer to focus on a different time-scale.
Our design is biologically inspired and draws on the architecture of the human brain which executes actions at different timescales depending on the environment's demands.
- Score: 2.1700103865910503
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: We present temporally layered architecture (TLA), a biologically inspired
system for temporally adaptive distributed control. TLA layers a fast and a
slow controller together to achieve temporal abstraction that allows each layer
to focus on a different time-scale. Our design is biologically inspired and
draws on the architecture of the human brain which executes actions at
different timescales depending on the environment's demands. Such distributed
control design is widespread across biological systems because it increases
survivability and accuracy in certain and uncertain environments. We
demonstrate that TLA can provide many advantages over existing approaches,
including persistent exploration, adaptive control, explainable temporal
behavior, compute efficiency and distributed control. We present two different
algorithms for training TLA: (a) Closed-loop control, where the fast controller
is trained over a pre-trained slow controller, allowing better exploration for
the fast controller and closed-loop control where the fast controller decides
whether to "act-or-not" at each timestep; and (b) Partially open loop control,
where the slow controller is trained over a pre-trained fast controller,
allowing for open loop-control where the slow controller picks a temporally
extended action or defers the next n-actions to the fast controller. We
evaluated our method on a suite of continuous control tasks and demonstrate the
advantages of TLA over several strong baselines.
Related papers
- Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency.
In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution.
Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
arXiv Detail & Related papers (2024-04-05T17:58:37Z) - Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal
Locomotion Control [112.66677641636299]
This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots.
We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.
This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
arXiv Detail & Related papers (2024-01-30T10:48:43Z) - Learning to Fly in Seconds [7.259696592534715]
We show how curriculum learning and a highly optimized simulator enhance sample complexity and lead to fast training times.
Our framework enables Simulation-to-Reality (Sim2Real) transfer for direct control after only 18 seconds of training on a consumer-grade laptop.
arXiv Detail & Related papers (2023-11-22T01:06:45Z) - Temporally Layered Architecture for Efficient Continuous Control [1.933681537640272]
We present a temporally layered architecture (TLA) for temporally adaptive control with minimal energy expenditure.
Our design draws on the energy-saving mechanism of the human brain, which executes actions at different timescales depending on the environment's demands.
arXiv Detail & Related papers (2023-05-30T02:59:06Z) - Deluca -- A Differentiable Control Library: Environments, Methods, and
Benchmarking [52.44199258132215]
We present an open-source library of differentiable physics and robotics environments.
The library features several popular environments, including classical control settings from OpenAI Gym.
We give several use-cases of new scientific results obtained using the library.
arXiv Detail & Related papers (2021-02-19T15:06:47Z) - Machine Learning for Mechanical Ventilation Control [52.65490904484772]
We consider the problem of controlling an invasive mechanical ventilator for pressure-controlled ventilation.
A PID controller must let air in and out of a sedated patient's lungs according to a trajectory of airway pressures specified by a clinician.
We show that our controllers are able to track target pressure waveforms significantly better than PID controllers.
arXiv Detail & Related papers (2021-02-12T21:23:33Z) - Regularizing Action Policies for Smooth Control with Reinforcement
Learning [47.312768123967025]
Conditioning for Action Policy Smoothness (CAPS) is an effective yet intuitive regularization on action policies.
CAPS offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers.
Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption.
arXiv Detail & Related papers (2020-12-11T21:35:24Z) - Learning a Contact-Adaptive Controller for Robust, Efficient Legged
Locomotion [95.1825179206694]
We present a framework that synthesizes robust controllers for a quadruped robot.
A high-level controller learns to choose from a set of primitives in response to changes in the environment.
A low-level controller that utilizes an established control method to robustly execute the primitives.
arXiv Detail & Related papers (2020-09-21T16:49:26Z) - Optimal PID and Antiwindup Control Design as a Reinforcement Learning
Problem [3.131740922192114]
We focus on the interpretability of DRL control methods.
In particular, we view linear fixed-structure controllers as shallow neural networks embedded in the actor-critic framework.
arXiv Detail & Related papers (2020-05-10T01:05:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.