Rethinking Optimization with Differentiable Simulation from a Global Perspective

• URL: http://arxiv.org/abs/2207.00167v1
• Date: Tue, 28 Jun 2022 17:08:53 GMT
• Title: Rethinking Optimization with Differentiable Simulation from a Global Perspective
• Authors: Rika Antonova, Jingyun Yang, Krishna Murthy Jatavallabhula, Jeannette Bohg
• Abstract summary: Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and system identification. We study the challenges that differentiable simulation presents when it is not feasible to expect that a single descent reaches a global optimum. We propose a method that combines Bayesian optimization with semi-local 'leaps' to obtain a global search method that can use gradients effectively.
• Score: 20.424212055832676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and system identification. However, existing approaches to differentiable simulation have largely tackled scenarios where obtaining smooth gradients has been relatively easy, such as systems with mostly smooth dynamics. In this work, we study the challenges that differentiable simulation presents when it is not feasible to expect that a single descent reaches a global optimum, which is often a problem in contact-rich scenarios. We analyze the optimization landscapes of diverse scenarios that contain both rigid bodies and deformable objects. In dynamic environments with highly deformable objects and fluids, differentiable simulators produce rugged landscapes with nonetheless useful gradients in some parts of the space. We propose a method that combines Bayesian optimization with semi-local 'leaps' to obtain a global search method that can use gradients effectively, while also maintaining robust performance in regions with noisy gradients. We show that our approach outperforms several gradient-based and gradient-free baselines on an extensive set of experiments in simulation, and also validate the method using experiments with a real robot and deformables. Videos and supplementary materials are available at https://tinyurl.com/globdiff

Related papers

• Improving Gradient Computation for Differentiable Physics Simulation with Contacts [10.450509067356148]
  We study differentiable rigid-body simulation with contacts. We propose to improve gradient computation by continuous collision detection and leverage the time-of-impact (TOI) We show that with TOI-Ve, we are able to learn an optimal control sequence that matches the analytical solution.
  arXiv  Detail & Related papers  (2023-04-28T21:10:16Z)
• FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation [80.63838153351804]
  We introduce FluidLab, a simulation environment with a diverse set of manipulation tasks involving complex fluid dynamics. At the heart of our platform is a fully differentiable physics simulator, providing GPU-accelerated simulations and gradient calculations. We propose several domain-specific optimization schemes coupled with differentiable physics.
  arXiv  Detail & Related papers  (2023-03-04T07:24:22Z)
• Near-optimal Policy Identification in Active Reinforcement Learning [84.27592560211909]
  AE-LSVI is a novel variant of the kernelized least-squares value RL (LSVI) algorithm that combines optimism with pessimism for active exploration. We show that AE-LSVI outperforms other algorithms in a variety of environments when robustness to the initial state is required.
  arXiv  Detail & Related papers  (2022-12-19T14:46:57Z)
• Differentiable Physics Simulations with Contacts: Do They Have Correct Gradients w.r.t. Position, Velocity and Control? [9.883261192383612]
  By making physics simulations end-to-end differentiable, we can perform gradient-based optimization and learning tasks. A majority of differentiable simulators consider collisions and contacts between objects, but they use different contact models for differentiability. In this paper, we overview four kinds of differentiable contact formulations - linear complementarity problems (LCP), convex optimization models, compliant models and position-based dynamics (PBD)
  arXiv  Detail & Related papers  (2022-07-08T19:30:09Z)
• DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects [18.266002992029716]
  Research in manipulation of deformable objects is typically conducted on a limited range of scenarios. Realistic simulators with support for various types of deformations and interactions have the potential to speed up experimentation. For highly deformable objects it is challenging to align the output of a simulator with the behavior of real objects.
  arXiv  Detail & Related papers  (2022-04-07T00:45:26Z)
• DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools [96.38972082580294]
  DiffSkill is a novel framework that uses a differentiable physics simulator for skill abstraction to solve deformable object manipulation tasks. In particular, we first obtain short-horizon skills using individual tools from a gradient-based simulator. We then learn a neural skill abstractor from the demonstration trajectories which takes RGBD images as input.
  arXiv  Detail & Related papers  (2022-03-31T17:59:38Z)
• Efficient Differentiable Simulation of Articulated Bodies [89.64118042429287]
  We present a method for efficient differentiable simulation of articulated bodies. This enables integration of articulated body dynamics into deep learning frameworks. We show that reinforcement learning with articulated systems can be accelerated using gradients provided by our method.
  arXiv  Detail & Related papers  (2021-09-16T04:48:13Z)
• PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics [89.81550748680245]
  We introduce a new differentiable physics benchmark called PasticineLab. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark.
  arXiv  Detail & Related papers  (2021-04-07T17:59:23Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• Black-Box Optimization with Local Generative Surrogates [6.04055755718349]
  In fields such as physics and engineering, many processes are modeled with non-differentiable simulators with intractable likelihoods. We introduce the use of deep generative models to approximate the simulator in local neighborhoods of the parameter space. In cases where the dependence of the simulator on the parameter space is constrained to a low dimensional submanifold, we observe that our method attains minima faster than baseline methods.
  arXiv  Detail & Related papers  (2020-02-11T19:02:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.