Learning by Doing: Controlling a Dynamical System using Causality,
Control, and Reinforcement Learning
- URL: http://arxiv.org/abs/2202.06052v1
- Date: Sat, 12 Feb 2022 12:37:29 GMT
- Title: Learning by Doing: Controlling a Dynamical System using Causality,
Control, and Reinforcement Learning
- Authors: Sebastian Weichwald, S{\o}ren Wengel Mogensen, Tabitha Edith Lee,
Dominik Baumann, Oliver Kroemer, Isabelle Guyon, Sebastian Trimpe, Jonas
Peters, Niklas Pfister
- Abstract summary: Questions in causality, control, and reinforcement learning go beyond the classical machine learning task of prediction.
We believe that combining the different views might create synergies and this competition is meant as a first step toward such synergies.
The goal in both tracks is to infer controls that drive the system to a desired state.
- Score: 27.564435351371653
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Questions in causality, control, and reinforcement learning go beyond the
classical machine learning task of prediction under i.i.d. observations.
Instead, these fields consider the problem of learning how to actively perturb
a system to achieve a certain effect on a response variable. Arguably, they
have complementary views on the problem: In control, one usually aims to first
identify the system by excitation strategies to then apply model-based design
techniques to control the system. In (non-model-based) reinforcement learning,
one directly optimizes a reward. In causality, one focus is on identifiability
of causal structure. We believe that combining the different views might create
synergies and this competition is meant as a first step toward such synergies.
The participants had access to observational and (offline) interventional data
generated by dynamical systems. Track CHEM considers an open-loop problem in
which a single impulse at the beginning of the dynamics can be set, while Track
ROBO considers a closed-loop problem in which control variables can be set at
each time step. The goal in both tracks is to infer controls that drive the
system to a desired state. Code is open-sourced (
https://github.com/LearningByDoingCompetition/learningbydoing-comp ) to
reproduce the winning solutions of the competition and to facilitate trying out
new methods on the competition tasks.
Related papers
- A Generative Approach to Control Complex Physical Systems [16.733151963652244]
We introduce Diffusion Physical systems Control (DiffPhyCon), a new class of method to address the physical systems control problem.
DiffPhyCon excels by simultaneously minimizing both the learned generative energy function and the predefined control objectives.
We test our method in 1D Burgers' equation and 2D jellyfish movement control in a fluid environment.
arXiv Detail & Related papers (2024-07-09T01:56:23Z) - Learning System Dynamics without Forgetting [60.08612207170659]
Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology.
We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics.
We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
arXiv Detail & Related papers (2024-06-30T14:55:18Z) - Controlling dynamical systems to complex target states using machine
learning: next-generation vs. classical reservoir computing [68.8204255655161]
Controlling nonlinear dynamical systems using machine learning allows to drive systems into simple behavior like periodicity but also to more complex arbitrary dynamics.
We show first that classical reservoir computing excels at this task.
In a next step, we compare those results based on different amounts of training data to an alternative setup, where next-generation reservoir computing is used instead.
It turns out that while delivering comparable performance for usual amounts of training data, next-generation RC significantly outperforms in situations where only very limited data is available.
arXiv Detail & Related papers (2023-07-14T07:05:17Z) - CT-DQN: Control-Tutored Deep Reinforcement Learning [4.395396671038298]
Control-Tutored Deep Q-Networks (CT-DQN) is a Deep Reinforcement Learning algorithm that leverages a control tutor to reduce learning time.
We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing.
arXiv Detail & Related papers (2022-12-02T17:59:43Z) - The least-control principle for learning at equilibrium [65.2998274413952]
We present a new principle for learning equilibrium recurrent neural networks, deep equilibrium models, or meta-learning.
Our results shed light on how the brain might learn and offer new ways of approaching a broad class of machine learning problems.
arXiv Detail & Related papers (2022-07-04T11:27:08Z) - Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions.
For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions.
The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
arXiv Detail & Related papers (2022-05-13T21:09:41Z) - Collision-Free Flocking with a Dynamic Squad of Fixed-Wing UAVs Using
Deep Reinforcement Learning [2.555094847583209]
We deal with the decentralized leader-follower flocking control problem through deep reinforcement learning (DRL)
We propose a novel reinforcement learning algorithm CACER-II for training a shared control policy for all the followers.
As a result, the variable-length system state can be encoded into a fixed-length embedding vector, which makes the learned DRL policies independent with the number or the order of followers.
arXiv Detail & Related papers (2021-01-20T11:23:35Z) - Machine Learning in Event-Triggered Control: Recent Advances and Open
Issues [0.7699714865575188]
This article reviews the literature on the use of machine learning in combination with event-triggered control.
We discuss how these learning algorithms can be used for different applications depending on the purpose of the machine learning use.
arXiv Detail & Related papers (2020-09-27T08:11:34Z) - Active Learning of Causal Structures with Deep Reinforcement Learning [13.202747831999414]
We study the problem of experiment design to learn causal structures from interventional data.
We present the first deep reinforcement learning based solution for the problem of experiment design.
arXiv Detail & Related papers (2020-09-07T10:49:06Z) - Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
AntLer is a design algorithm for learning-based control laws that anticipates learning.
We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
arXiv Detail & Related papers (2020-07-24T07:00:14Z) - Data-driven Koopman Operators for Model-based Shared Control of
Human-Machine Systems [66.65503164312705]
We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex machines.
Both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator.
We find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.
arXiv Detail & Related papers (2020-06-12T14:14:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.