QuadSim: A Quadcopter Rotational Dynamics Simulation Framework For Reinforcement Learning Algorithms

• URL: http://arxiv.org/abs/2202.07021v1
• Date: Mon, 14 Feb 2022 20:34:08 GMT
• Title: QuadSim: A Quadcopter Rotational Dynamics Simulation Framework For Reinforcement Learning Algorithms
• Authors: Burak Han Demirbilek
• Abstract summary: This study focuses on designing and developing a mathematically based quadcopter rotational dynamics simulation framework. The framework aims to simulate both linear and nonlinear representations of a quadcopter. The simulation environment has been expanded to be compatible with the OpenAI Gym toolkit.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study focuses on designing and developing a mathematically based quadcopter rotational dynamics simulation framework for testing reinforcement learning (RL) algorithms in many flexible configurations. The design of the simulation framework aims to simulate both linear and nonlinear representations of a quadcopter by solving initial value problems for ordinary differential equation (ODE) systems. In addition, the simulation environment is capable of making the simulation deterministic/stochastic by adding random Gaussian noise in the forms of process and measurement noises. In order to ensure that the scope of this simulation environment is not limited only with our own RL algorithms, the simulation environment has been expanded to be compatible with the OpenAI Gym toolkit. The framework also supports multiprocessing capabilities to run simulation environments simultaneously in parallel. To test these capabilities, many state-of-the-art deep RL algorithms were trained in this simulation framework and the results were compared in detail.

Related papers

• GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator [55.02281855589641]
  GauSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels. We leverage continuum mechanics, modeling each kernel as a continuous piece of matter to account for realistic deformations without idealized assumptions. GauSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation [11.360832156847103]
  This paper presents a novel RL algorithm and a simulation platform to enable scaling RL on tasks involving rigid bodies and deformables. We introduce Soft Analytic Policy (SAPO), a maximum entropy first-order model-based RL algorithm, which uses first-order analytic gradients from differentiable simulation to train an actor to maximize expected return and entropy. We also develop Rewarped, a parallel differentiable multiphysics simulation platform that supports simulating various materials beyond rigid bodies.
  arXiv  Detail & Related papers  (2024-12-16T18:56:24Z)
• Learning Quadruped Locomotion Using Differentiable Simulation [31.80380408663424]
  Differentiable simulation promises fast convergence and stable training. This work proposes a new differentiable simulation framework to overcome these challenges. Our framework enables learning quadruped walking in simulation in minutes without parallelization.
  arXiv  Detail & Related papers  (2024-03-21T22:18:59Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Physical Systems Modeled Without Physical Laws [0.0]
  Tree-based machine learning methods can emulate desired outputs without "knowing" the complex backing involved in the simulations. We specifically focus on predicting specific spatial-temporal data between two simulation outputs and increasing spatial resolution to generalize the physics predictions to finer test grids without the computational costs of repeating the numerical calculation.
  arXiv  Detail & Related papers  (2022-07-26T20:51:20Z)
• Accelerated Policy Learning with Parallel Differentiable Simulation [59.665651562534755]
  We present a differentiable simulator and a new policy learning algorithm (SHAC) Our algorithm alleviates problems with local minima through a smooth critic function. We show substantial improvements in sample efficiency and wall-clock time over state-of-the-art RL and differentiable simulation-based algorithms.
  arXiv  Detail & Related papers  (2022-04-14T17:46:26Z)
• ElVibRot-MPI: parallel quantum dynamics with Smolyak algorithm for general molecular simulation [0.0]
  A parallelized quantum dynamics package using the Smolyak algorithm for general molecular simulation is introduced. The program has no limitation of the Hamiltonian form and provides high flexibility on the simulation setup to adapt to different problems.
  arXiv  Detail & Related papers  (2021-11-26T18:34:07Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• SimNet: Computer Architecture Simulation using Machine Learning [3.7019798164954336]
  This work describes a concerted effort, where machine learning (ML) is used to accelerate discrete-event simulation. A GPU-accelerated parallel simulator is implemented based on the proposed instruction latency predictor. Its simulation accuracy and throughput are validated and evaluated against a state-of-the-art simulator.
  arXiv  Detail & Related papers  (2021-05-12T17:31:52Z)
• Reinforcement Learning for Adaptive Mesh Refinement [63.7867809197671]
  We propose a novel formulation of AMR as a Markov decision process and apply deep reinforcement learning to train refinement policies directly from simulation. The model sizes of these policy architectures are independent of the mesh size and hence scale to arbitrarily large and complex simulations.
  arXiv  Detail & Related papers  (2021-03-01T22:55:48Z)
• A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
  This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
  arXiv  Detail & Related papers  (2020-11-17T22:24:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.