Learning Optimal Decision Making for an Industrial Truck Unloading Robot using Minimal Simulator Runs

• URL: http://arxiv.org/abs/2105.05019v1
• Date: Sat, 13 Mar 2021 06:22:23 GMT
• Title: Learning Optimal Decision Making for an Industrial Truck Unloading Robot using Minimal Simulator Runs
• Authors: Manash Pratim Das, Anirudh Vemula, Mayank Pathak, Sandip Aine, Maxim Likhachev
• Abstract summary: Most high-fidelity robotic simulators like ours are time-consuming. For the truck unloading problem, our experiments show that a significant reduction in simulator runs can be achieved.
• Score: 15.208516853395453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Consider a truck filled with boxes of varying size and unknown mass and an industrial robot with end-effectors that can unload multiple boxes from any reachable location. In this work, we investigate how would the robot with the help of a simulator, learn to maximize the number of boxes unloaded by each action. Most high-fidelity robotic simulators like ours are time-consuming. Therefore, we investigate the above learning problem with a focus on minimizing the number of simulation runs required. The optimal decision-making problem under this setting can be formulated as a multi-class classification problem. However, to obtain the outcome of any action requires us to run the time-consuming simulator, thereby restricting the amount of training data that can be collected. Thus, we need a data-efficient approach to learn the classifier and generalize it with a minimal amount of data. A high-fidelity physics-based simulator is common in general for complex manipulation tasks involving multi-body interactions. To this end, we train an optimal decision tree as the classifier, and for each branch of the decision tree, we reason about the confidence in the decision using a Probably Approximately Correct (PAC) framework to determine whether more simulator data will help reach a certain confidence level. This provides us with a mechanism to evaluate when simulation can be avoided for certain decisions, and when simulation will improve the decision making. For the truck unloading problem, our experiments show that a significant reduction in simulator runs can be achieved using the proposed method as compared to naively running the simulator to collect data to train equally performing decision trees.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Continual learning autoencoder training for a particle-in-cell simulation via streaming [52.77024349608834]
  upcoming exascale era will provide a new generation of physics simulations with high resolution. These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
  arXiv  Detail & Related papers  (2022-11-09T09:55:14Z)
• Obstacle Avoidance for Robotic Manipulator in Joint Space via Improved Proximal Policy Optimization [6.067589886362815]
  In this paper, we train a deep neural network via an improved Proximal Policy Optimization (PPO) algorithm to map from task space to joint space for a 6-DoF manipulator. Since training such a task in real-robot is time-consuming and strenuous, we develop a simulation environment to train the model. Experimental results showed that using our method, the robot was capable of tracking a single target or reaching multiple targets in unstructured environments.
  arXiv  Detail & Related papers  (2022-10-03T10:21:57Z)
• DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cutting [71.50844437057555]
  We present DiSECt: the first differentiable simulator for cutting soft materials. The simulator augments the finite element method with a continuous contact model based on signed distance fields. We show that the simulator can be calibrated to match resultant forces and fields from a state-of-the-art commercial solver.
  arXiv  Detail & Related papers  (2022-03-19T07:27:19Z)
• Online Planning in POMDPs with Self-Improving Simulators [17.722070992253638]
  We learn online an approximate but much faster simulator that improves over time. To plan reliably and efficiently while the approximate simulator is learning, we develop a method that adaptively decides which simulator to use for every simulation. Experimental results in two large domains show that when integrated with POMCP, our approach allows to plan with improving efficiency over time.
  arXiv  Detail & Related papers  (2022-01-27T09:41:59Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• Robot Learning from Randomized Simulations: A Review [59.992761565399185]
  Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive. We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
  arXiv  Detail & Related papers  (2021-11-01T13:55:41Z)
• An Active Learning Framework for Constructing High-fidelity Mobility Maps [0.0]
  We introduce an active learning paradigm that substantially reduces the number of simulations needed to train a machine learning classifier without sacrificing accuracy. Experimental results suggest that our sampling algorithm can train a neural network, with higher accuracy, using less than half the number of simulations when compared to random sampling.
  arXiv  Detail & Related papers  (2020-03-07T04:50:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.