Data-Efficient Learning for Complex and Real-Time Physical Problem
Solving using Augmented Simulation
- URL: http://arxiv.org/abs/2011.07193v2
- Date: Tue, 16 Feb 2021 02:30:59 GMT
- Title: Data-Efficient Learning for Complex and Real-Time Physical Problem
Solving using Augmented Simulation
- Authors: Kei Ota, Devesh K. Jha, Diego Romeres, Jeroen van Baar, Kevin A.
Smith, Takayuki Semitsu, Tomoaki Oiki, Alan Sullivan, Daniel Nikovski, and
Joshua B. Tenenbaum
- Abstract summary: We present a task for navigating a marble to the center of a circular maze.
We present a model that learns to move a marble in the complex environment within minutes of interacting with the real system.
- Score: 49.631034790080406
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Humans quickly solve tasks in novel systems with complex dynamics, without
requiring much interaction. While deep reinforcement learning algorithms have
achieved tremendous success in many complex tasks, these algorithms need a
large number of samples to learn meaningful policies. In this paper, we present
a task for navigating a marble to the center of a circular maze. While this
system is very intuitive and easy for humans to solve, it can be very difficult
and inefficient for standard reinforcement learning algorithms to learn
meaningful policies. We present a model that learns to move a marble in the
complex environment within minutes of interacting with the real system.
Learning consists of initializing a physics engine with parameters estimated
using data from the real system. The error in the physics engine is then
corrected using Gaussian process regression, which is used to model the
residual between real observations and physics engine simulations. The physics
engine augmented with the residual model is then used to control the marble in
the maze environment using a model-predictive feedback over a receding horizon.
To the best of our knowledge, this is the first time that a hybrid model
consisting of a full physics engine along with a statistical function
approximator has been used to control a complex physical system in real-time
using nonlinear model-predictive control (NMPC).
Related papers
- Physics Encoded Blocks in Residual Neural Network Architectures for Digital Twin Models [2.8720819157502344]
This paper presents a generic approach based on a novel physics-encoded residual neural network architecture.
Our method combines physics blocks as mathematical operators from physics-based models with learning blocks comprising feed-forward layers.
Compared to conventional neural network-based methods, our method improves generalizability with substantially low data requirements.
arXiv Detail & Related papers (2024-11-18T11:58:20Z) - Reduced Simulations for High-Energy Physics, a Middle Ground for
Data-Driven Physics Research [0.0]
Subatomic particle track reconstruction is a vital task in High-Energy Physics experiments.
We provide the REDuced VIrtual Detector (REDVID) as a complexity-reduced detector model and particle collision event simulator combo.
arXiv Detail & Related papers (2023-08-30T12:50:45Z) - Learning Controllable Adaptive Simulation for Multi-resolution Physics [86.8993558124143]
We introduce Learning controllable Adaptive simulation for Multi-resolution Physics (LAMP) as the first full deep learning-based surrogate model.
LAMP consists of a Graph Neural Network (GNN) for learning the forward evolution, and a GNN-based actor-critic for learning the policy of spatial refinement and coarsening.
We demonstrate that our LAMP outperforms state-of-the-art deep learning surrogate models, and can adaptively trade-off computation to improve long-term prediction error.
arXiv Detail & Related papers (2023-05-01T23:20:27Z) - Hindsight States: Blending Sim and Real Task Elements for Efficient
Reinforcement Learning [61.3506230781327]
In robotics, one approach to generate training data builds on simulations based on dynamics models derived from first principles.
Here, we leverage the imbalance in complexity of the dynamics to learn more sample-efficiently.
We validate our method on several challenging simulated tasks and demonstrate that it improves learning both alone and when combined with an existing hindsight algorithm.
arXiv Detail & Related papers (2023-03-03T21:55:04Z) - Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse
Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
We learn the residual errors between a dynamic and/or simulator model and the real robot.
We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
arXiv Detail & Related papers (2022-09-07T15:15:12Z) - Automated Dissipation Control for Turbulence Simulation with Shell
Models [1.675857332621569]
The application of machine learning (ML) techniques, especially neural networks, has seen tremendous success at processing images and language.
In this work we construct a strongly simplified representation of turbulence by using the Gledzer-Ohkitani-Yamada shell model.
We propose an approach that aims to reconstruct statistical properties of turbulence such as the self-similar inertial-range scaling.
arXiv Detail & Related papers (2022-01-07T15:03:52Z) - PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable
Physics [89.81550748680245]
We introduce a new differentiable physics benchmark called PasticineLab.
In each task, the agent uses manipulators to deform the plasticine into the desired configuration.
We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark.
arXiv Detail & Related papers (2021-04-07T17:59:23Z) - Fast Modeling and Understanding Fluid Dynamics Systems with
Encoder-Decoder Networks [0.0]
We show that an accurate deep-learning-based proxy model can be taught efficiently by a finite-volume-based simulator.
Compared to traditional simulation, the proposed deep learning approach enables much faster forward computation.
We quantify the sensitivity of the deep learning model to key physical parameters and hence demonstrate that the inversion problems can be solved with great acceleration.
arXiv Detail & Related papers (2020-06-09T17:14:08Z) - Information Theoretic Model Predictive Q-Learning [64.74041985237105]
We present a novel theoretical connection between information theoretic MPC and entropy regularized RL.
We develop a Q-learning algorithm that can leverage biased models.
arXiv Detail & Related papers (2019-12-31T00:29:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.