Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materials

• URL: http://arxiv.org/abs/2411.00554v3
• Date: Tue, 18 Feb 2025 08:35:55 GMT
• Title: Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materials
• Authors: Xintong Yang, Ze Ji, Yu-Kun Lai,
• Abstract summary: This work introduces a novel Differentiable Physics-based System Identification (DPSI) framework that enables a robot arm to infer the physics parameters of elastoplastic materials and the environment using simple manipulation motions and incomplete 3D point clouds. With only a single real-world interaction, the estimated parameters can accurately simulate visually and physically realistic behaviours induced by unseen and long-horizon manipulation motions.
• Score: 43.99845081513279
• License:
• Abstract: Robotic manipulation of volumetric elastoplastic deformable materials, from foods such as dough to construction materials like clay, is in its infancy, largely due to the difficulty of modelling and perception in a high-dimensional space. Simulating the dynamics of such materials is computationally expensive. It tends to suffer from inaccurately estimated physics parameters of the materials and the environment, impeding high-precision manipulation. Estimating such parameters from raw point clouds captured by optical cameras suffers further from heavy occlusions. To address this challenge, this work introduces a novel Differentiable Physics-based System Identification (DPSI) framework that enables a robot arm to infer the physics parameters of elastoplastic materials and the environment using simple manipulation motions and incomplete 3D point clouds, aligning the simulation with the real world. Extensive experiments show that with only a single real-world interaction, the estimated parameters, Young's modulus, Poisson's ratio, yield stress and friction coefficients, can accurately simulate visually and physically realistic deformation behaviours induced by unseen and long-horizon manipulation motions. Additionally, the DPSI framework inherently provides physically intuitive interpretations for the parameters in contrast to black-box approaches such as deep neural networks. The project is fully open-sourced via https://ianyangchina.github.io/SI4RP-data/.

Related papers

• LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models [35.01842161084472]
  We propose a new physical reasoning task and a dataset, dubbed TraySim. Our task involves predicting the dynamics of several objects on a tray that is given an external impact. We present LLMPhy, a zero-shot black-box optimization framework that leverages the physics knowledge and program synthesis abilities of LLMs. Our results show that the combination of the LLM and the physics engine leads to state-of-the-art zero-shot physical reasoning performance.
  arXiv  Detail & Related papers  (2024-11-12T18:56:58Z)
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion [35.71595369663293]
  We propose textbfPhysics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model. Experiments demonstrate the effectiveness of our method with both elastic and plastic materials.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• PhyRecon: Physically Plausible Neural Scene Reconstruction [81.73129450090684]
  We introduce PHYRECON, the first approach to leverage both differentiable rendering and differentiable physics simulation to learn implicit surface representations. Central to this design is an efficient transformation between SDF-based implicit representations and explicit surface points. Our results also exhibit superior physical stability in physical simulators, with at least a 40% improvement across all datasets.
  arXiv  Detail & Related papers  (2024-04-25T15:06:58Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• Physics-Enhanced Multi-fidelity Learning for Optical Surface Imprint [1.0878040851638]
  We propose a novel method to use multi-fidelity neural networks (MFNN) to solve this inverse problem. We build up the NN model via pure simulation data, and then bridge the sim-to-real gap via transfer learning. Considering the difficulty of collecting real experimental data, we use NN to dig out the unknown physics and also implant the known physics into the transfer learning framework.
  arXiv  Detail & Related papers  (2023-11-17T01:55:15Z)
• PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification [64.61198351207752]
  Existing approaches to system identification (estimating the physical parameters of an object) from videos assume known object geometries. In this work, we aim to identify parameters characterizing a physical system from a set of multi-view videos without any assumption on object geometry or topology. We propose "Physics Augmented Continuum Neural Radiance Fields" (PAC-NeRF), to estimate both the unknown geometry and physical parameters of highly dynamic objects from multi-view videos.
  arXiv  Detail & Related papers  (2023-03-09T18:59:50Z)
• Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language [92.7638697243969]
  We propose a unified framework that can jointly learn visual concepts and infer physics models of objects from videos and language. This is achieved by seamlessly integrating three components: a visual perception module, a concept learner, and a differentiable physics engine.
  arXiv  Detail & Related papers  (2021-10-28T17:59:13Z)
• Learning Material Parameters and Hydrodynamics of Soft Robotic Fish via Differentiable Simulation [26.09104786491426]
  Our framework allows high fidelity prediction of dynamic behavior for composite bi-morph bending structures in real hardware. We demonstrate an experimentally-verified, fast optimization pipeline for learning the material parameters and hydrodynamics of our robots. Although we focus on a specific application for underwater soft robots, our framework is applicable to any pneumatically actuated soft mechanism.
  arXiv  Detail & Related papers  (2021-09-30T05:24:02Z)
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling [86.9726984929758]
  We focus on the integration of incomplete physics models into deep generative models. We propose a VAE architecture in which a part of the latent space is grounded by physics. We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-25T20:28:52Z)
• Gaining a Sense of Touch. Physical Parameters Estimation using a Soft Gripper and Neural Networks [3.0892724364965005]
  There is not enough research on physical parameters estimation using deep learning algorithms on measurements from direct interaction with objects using robotic grippers. We propose a trainable system for the regression of a stiffness coefficient and provided extensive experiments using the physics simulator environment. Our system can reliably estimate the stiffness of an object using the Yale OpenHand soft gripper based on readings from Inertial Measurement Units (IMUs) attached to its fingers.
  arXiv  Detail & Related papers  (2020-03-02T11:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.