PhyPlan: Compositional and Adaptive Physical Task Reasoning with Physics-Informed Skill Networks for Robot Manipulators

• URL: http://arxiv.org/abs/2402.15767v1
• Date: Sat, 24 Feb 2024 08:51:03 GMT
• Title: PhyPlan: Compositional and Adaptive Physical Task Reasoning with Physics-Informed Skill Networks for Robot Manipulators
• Authors: Harshil Vagadia and Mudit Chopra and Abhinav Barnawal and Tamajit Banerjee and Shreshth Tuli and Souvik Chakraborty and Rohan Paul
• Abstract summary: Existing methods for physical reasoning are data-hungry and struggle with complexity and uncertainty inherent in the real world. This paper presents PhyPlan, a physics-informed planning framework that combines physics-informed neural networks (PINNs) with modified Monte Carlo Tree Search (MCTS) to enable embodied agents to perform dynamic physical tasks.
• Score: 5.680235630702706
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Given the task of positioning a ball-like object to a goal region beyond direct reach, humans can often throw, slide, or rebound objects against the wall to attain the goal. However, enabling robots to reason similarly is non-trivial. Existing methods for physical reasoning are data-hungry and struggle with complexity and uncertainty inherent in the real world. This paper presents PhyPlan, a novel physics-informed planning framework that combines physics-informed neural networks (PINNs) with modified Monte Carlo Tree Search (MCTS) to enable embodied agents to perform dynamic physical tasks. PhyPlan leverages PINNs to simulate and predict outcomes of actions in a fast and accurate manner and uses MCTS for planning. It dynamically determines whether to consult a PINN-based simulator (coarse but fast) or engage directly with the actual environment (fine but slow) to determine optimal policy. Evaluation with robots in simulated 3D environments demonstrates the ability of our approach to solve 3D-physical reasoning tasks involving the composition of dynamic skills. Quantitatively, PhyPlan excels in several aspects: (i) it achieves lower regret when learning novel tasks compared to state-of-the-art, (ii) it expedites skill learning and enhances the speed of physical reasoning, (iii) it demonstrates higher data efficiency compared to a physics un-informed approach.

Related papers

• LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models [35.01842161084472]
  We propose a new physical reasoning task and a dataset, dubbed TraySim. Our task involves predicting the dynamics of several objects on a tray that is given an external impact. We present LLMPhy, a zero-shot black-box optimization framework that leverages the physics knowledge and program synthesis abilities of LLMs. Our results show that the combination of the LLM and the physics engine leads to state-of-the-art zero-shot physical reasoning performance.
  arXiv  Detail & Related papers  (2024-11-12T18:56:58Z)
• PhyPlan: Generalizable and Rapid Physical Task Planning with Physics Informed Skill Networks for Robot Manipulators [5.4089975505600005]
  Existing methods for physical reasoning are data-hungry and struggle with complexity and uncertainty inherent in the real world. This paper presents PhyPlan, a physics-informed planning framework that combines physics-informed neural networks (PINNs) with modified Monte Carlo Tree Search (MCTS) to enable embodied agents to perform dynamic physical tasks.
  arXiv  Detail & Related papers  (2024-04-22T06:35:08Z)
• PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models [58.33913881592706]
  Humans can easily apply their intuitive physics to grasp skillfully and change grasps efficiently, even for objects they have never seen before. This work delves into infusing such physical commonsense reasoning into robotic manipulation. We introduce PhyGrasp, a multimodal large model that leverages inputs from two modalities: natural language and 3D point clouds.
  arXiv  Detail & Related papers  (2024-02-26T18:57:52Z)
• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• Physics-Based Task Generation through Causal Sequence of Physical Interactions [3.2244944291325996]
  Performing tasks in a physical environment is a crucial yet challenging problem for AI systems operating in the real world. We present a systematic approach for defining a physical scenario using a causal sequence of physical interactions between objects. We then propose a methodology for generating tasks in a physics-simulating environment using defined scenarios as inputs.
  arXiv  Detail & Related papers  (2023-08-05T10:15:18Z)
• Progressive Learning for Physics-informed Neural Motion Planning [1.9798034349981157]
  Motion planning is one of the core robotics problems requiring fast methods for finding a collision-free robot motion path. Recent advancements have led to a physics-informed NMP approach that directly solves the Eikonal equation for motion planning. This paper presents a novel and tractable Eikonal equation formulation and introduces a new progressive learning strategy to train neural networks without expert data.
  arXiv  Detail & Related papers  (2023-06-01T12:41:05Z)
• NTFields: Neural Time Fields for Physics-Informed Robot Motion Planning [1.9798034349981157]
  We propose Neural Time Fields (NTFields) for robot motion planning in cluttered scenarios. Our framework represents a wave propagation model generating continuous arrival time to find path solutions informed by a nonlinear first-order PDE called Eikonal Equation. We evaluate our method in various cluttered 3D environments, including the Gibson dataset, and demonstrate its ability to solve motion planning problems for 4-DOF and 6-DOF robot manipulators.
  arXiv  Detail & Related papers  (2022-09-30T22:34:54Z)
• Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction [158.74130075865835]
  Given a malfunctional 3D object, humans can perform mental simulations to reason about its functionality and figure out how to fix it. To mimic humans' mental simulation process, we present FixNet, a novel framework that seamlessly incorporates perception and physical dynamics.
  arXiv  Detail & Related papers  (2022-05-05T17:59:36Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics [89.81550748680245]
  We introduce a new differentiable physics benchmark called PasticineLab. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark.
  arXiv  Detail & Related papers  (2021-04-07T17:59:23Z)
• Predicting the Physical Dynamics of Unseen 3D Objects [65.49291702488436]
  We focus on predicting the dynamics of 3D objects on a plane that have just been subjected to an impulsive force. Our approach can generalize to object shapes and initial conditions that were unseen during training. Our model can support training with data from both a physics engine or the real world.
  arXiv  Detail & Related papers  (2020-01-16T06:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.