Grounding Graph Network Simulators using Physical Sensor Observations

• URL: http://arxiv.org/abs/2302.11864v1
• Date: Thu, 23 Feb 2023 09:06:42 GMT
• Title: Grounding Graph Network Simulators using Physical Sensor Observations
• Authors: Jonas Linkerh\"agner, Niklas Freymuth, Paul Maria Scheikl, Franziska Mathis-Ullrich, Gerhard Neumann
• Abstract summary: We integrate sensory information to ground Graph Network Simulators on real world observations. We predict the mesh state of deformable objects by utilizing point cloud data.
• Score: 12.017054986629846
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physical simulations that accurately model reality are crucial for many engineering disciplines such as mechanical engineering and robotic motion planning. In recent years, learned Graph Network Simulators produced accurate mesh-based simulations while requiring only a fraction of the computational cost of traditional simulators. Yet, the resulting predictors are confined to learning from data generated by existing mesh-based simulators and thus cannot include real world sensory information such as point cloud data. As these predictors have to simulate complex physical systems from only an initial state, they exhibit a high error accumulation for long-term predictions. In this work, we integrate sensory information to ground Graph Network Simulators on real world observations. In particular, we predict the mesh state of deformable objects by utilizing point cloud data. The resulting model allows for accurate predictions over longer time horizons, even under uncertainties in the simulation, such as unknown material properties. Since point clouds are usually not available for every time step, especially in online settings, we employ an imputation-based model. The model can make use of such additional information only when provided, and resorts to a standard Graph Network Simulator, otherwise. We experimentally validate our approach on a suite of prediction tasks for mesh-based interactions between soft and rigid bodies. Our method results in utilization of additional point cloud information to accurately predict stable simulations where existing Graph Network Simulators fail.

Related papers

• Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
  We present SIM-FSVGD for learning robot dynamics from data. We use low-fidelity physical priors to regularize the training of neural network models. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
  arXiv  Detail & Related papers  (2024-03-25T11:29:32Z)
• Sim2Real for Environmental Neural Processes [20.850715955359593]
  We analyse 'Sim2Real': pre-training on reanalysis and fine-tuning on observational data. Sim2Real could enable more accurate models for weather prediction and climate monitoring.
  arXiv  Detail & Related papers  (2023-10-30T18:49:06Z)
• Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level [1.1999555634662635]
  LiDAR object detection algorithms based on neural networks for autonomous driving require large amounts of data for training, validation, and testing. We show that using simulated data for the training of neural networks leads to a domain shift of training and testing data due to differences in scenes, scenarios, and distributions.
  arXiv  Detail & Related papers  (2023-03-03T12:52:01Z)
• Continual learning autoencoder training for a particle-in-cell simulation via streaming [52.77024349608834]
  upcoming exascale era will provide a new generation of physics simulations with high resolution. These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
  arXiv  Detail & Related papers  (2022-11-09T09:55:14Z)
• PCGen: Point Cloud Generator for LiDAR Simulation [10.692184635629792]
  Existing methods generate data which are more noisy and complete than the real point clouds. We propose FPA raycasting and surrogate model raydrop. With minimal training data, the surrogate model can generalize to different geographies and scenes. Results show that object detection models trained by simulation data can achieve similar result as the real data trained model.
  arXiv  Detail & Related papers  (2022-10-17T04:13:21Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Learning Large-scale Subsurface Simulations with a Hybrid Graph Network Simulator [57.57321628587564]
  We introduce Hybrid Graph Network Simulator (HGNS) for learning reservoir simulations of 3D subsurface fluid flows. HGNS consists of a subsurface graph neural network (SGNN) to model the evolution of fluid flows, and a 3D-U-Net to model the evolution of pressure. Using an industry-standard subsurface flow dataset (SPE-10) with 1.1 million cells, we demonstrate that HGNS is able to reduce the inference time up to 18 times compared to standard subsurface simulators.
  arXiv  Detail & Related papers  (2022-06-15T17:29:57Z)
• Constraint-based graph network simulator [9.462808515258464]
  We present a framework for constraint-based learned simulation. We implement our method using a graph neural network as the constraint function and gradient descent as the constraint solver. Our model achieves better or comparable performance to top learned simulators.
  arXiv  Detail & Related papers  (2021-12-16T19:15:11Z)
• Training robust anomaly detection using ML-Enhanced simulations [1.370633147306388]
  Simulations can provide edge conditions for anomaly detection which may be sparse or non-existent in real-world data. Our approach enhances simulations using neural networks trained on real-world data to create outputs that are more realistic and variable than traditional simulations.
  arXiv  Detail & Related papers  (2020-08-27T12:28:07Z)
• Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction [88.0416857308144]
  We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. We directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing.
  arXiv  Detail & Related papers  (2020-08-13T17:20:02Z)
• Learning to Simulate Complex Physics with Graph Networks [68.43901833812448]
  We present a machine learning framework and model implementation that can learn to simulate a wide variety of challenging physical domains. Our framework---which we term "Graph Network-based Simulators" (GNS)--represents the state of a physical system with particles, expressed as nodes in a graph, and computes dynamics via learned message-passing. Our results show that our model can generalize from single-timestep predictions with thousands of particles during training, to different initial conditions, thousands of timesteps, and at least an order of magnitude more particles at test time.
  arXiv  Detail & Related papers  (2020-02-21T16:44:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.