Localized Graph-Based Neural Dynamics Models for Terrain Manipulation

• URL: http://arxiv.org/abs/2503.23270v1
• Date: Sun, 30 Mar 2025 01:24:10 GMT
• Title: Localized Graph-Based Neural Dynamics Models for Terrain Manipulation
• Authors: Chaoqi Liu, Yunzhu Li, Kris Hauser,
• Abstract summary: This paper introduces a learning-based approach for terrain dynamics modeling and manipulation.<n>We leverage the Graph-based Neural Dynamics framework to represent terrain deformation as motion of a graph of particles.
• Score: 29.578534072345835
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predictive models can be particularly helpful for robots to effectively manipulate terrains in construction sites and extraterrestrial surfaces. However, terrain state representations become extremely high-dimensional especially to capture fine-resolution details and when depth is unknown or unbounded. This paper introduces a learning-based approach for terrain dynamics modeling and manipulation, leveraging the Graph-based Neural Dynamics (GBND) framework to represent terrain deformation as motion of a graph of particles. Based on the principle that the moving portion of a terrain is usually localized, our approach builds a large terrain graph (potentially millions of particles) but only identifies a very small active subgraph (hundreds of particles) for predicting the outcomes of robot-terrain interaction. To minimize the size of the active subgraph we introduce a learning-based approach that identifies a small region of interest (RoI) based on the robot's control inputs and the current scene. We also introduce a novel domain boundary feature encoding that allows GBNDs to perform accurate dynamics prediction in the RoI interior while avoiding particle penetration through RoI boundaries. Our proposed method is both orders of magnitude faster than naive GBND and it achieves better overall prediction accuracy. We further evaluated our framework on excavation and shaping tasks on terrain with different granularity.

Related papers

• Watch Your STEPP: Semantic Traversability Estimation using Pose Projected Features [4.392942391043664]
  We propose a method for estimating terrain traversability by learning from demonstrations of human walking.<n>Our approach leverages dense, pixel-wise feature embeddings generated using the DINOv2 vision Transformer model.<n>By minimizing loss, the network distinguishes between familiar terrain with a low reconstruction error and unfamiliar or hazardous terrain with a higher reconstruction error.
  arXiv  Detail & Related papers  (2025-01-29T11:53:58Z)
• ImplicitTerrain: a Continuous Surface Model for Terrain Data Analysis [14.013976303831313]
  ImplicitTerrain is an implicit neural representation (INR) approach for modeling high-resolution terrain continuously and differentiably. Our experiments demonstrate superior surface fitting accuracy, effective topological feature retrieval, and various topographical feature extraction.
  arXiv  Detail & Related papers  (2024-05-31T23:05:34Z)
• TK-Planes: Tiered K-Planes with High Dimensional Feature Vectors for Dynamic UAV-based Scenes [58.180556221044235]
  We present a new approach to bridge the domain gap between synthetic and real-world data for unmanned aerial vehicle (UAV)-based perception. Our formulation is designed for dynamic scenes, consisting of small moving objects or human actions. We evaluate its performance on challenging datasets, including Okutama Action and UG2.
  arXiv  Detail & Related papers  (2024-05-04T21:55:33Z)
• Terrain Diffusion Network: Climatic-Aware Terrain Generation with Geological Sketch Guidance [16.29267504093274]
  Sketch-based terrain generation seeks to create realistic landscapes for virtual environments in various applications such as computer games, animation and virtual reality. We propose a novel diffusion-based method, namely terrain diffusion network (TDN), which actively incorporates user guidance for enhanced controllability. Three terrain synthesisers are designed for structural, intermediate, and fine-grained level denoising purposes, which allow each synthesiser concentrate on a distinct terrain aspect.
  arXiv  Detail & Related papers  (2023-08-31T13:41:34Z)
• Local-Global Information Interaction Debiasing for Dynamic Scene Graph Generation [51.92419880088668]
  We propose a novel DynSGG model based on multi-task learning, DynSGG-MTL, which introduces the local interaction information and global human-action interaction information. Long-temporal human actions supervise the model to generate multiple scene graphs that conform to the global constraints and avoid the model being unable to learn the tail predicates.
  arXiv  Detail & Related papers  (2023-08-10T01:24:25Z)
• NEWTON: Neural View-Centric Mapping for On-the-Fly Large-Scale SLAM [51.21564182169607]
  Newton is a view-centric mapping method that dynamically constructs neural fields based on run-time observation. Our method enables camera pose updates using loop closures and scene boundary updates by representing the scene with multiple neural fields. The experimental results demonstrate the superior performance of our method over existing world-centric neural field-based SLAM systems.
  arXiv  Detail & Related papers  (2023-03-23T20:22:01Z)
• LOPR: Latent Occupancy PRediction using Generative Models [49.15687400958916]
  LiDAR generated occupancy grid maps (L-OGMs) offer a robust bird's eye-view scene representation. We propose a framework that decouples occupancy prediction into: representation learning and prediction within the learned latent space.
  arXiv  Detail & Related papers  (2022-10-03T22:04:00Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)
• Learning Multi-Object Dynamics with Compositional Neural Radiance Fields [63.424469458529906]
  We present a method to learn compositional predictive models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks. NeRFs have become a popular choice for representing scenes due to their strong 3D prior. For planning, we utilize RRTs in the learned latent space, where we can exploit our model and the implicit object encoder to make sampling the latent space informative and more efficient.
  arXiv  Detail & Related papers  (2022-02-24T01:31:29Z)
• Deep Generative Framework for Interactive 3D Terrain Authoring and Manipulation [4.202216894379241]
  We propose a novel realistic terrain authoring framework powered by a combination of VAE and generative conditional GAN model. Our framework is an example-based method that attempts to overcome the limitations of existing methods by learning a latent space from a real-world terrain dataset. We also developed an interactive tool, that lets the user generate diverse terrains with minimalist inputs.
  arXiv  Detail & Related papers  (2022-01-07T08:58:01Z)
• Solving Occlusion in Terrain Mapping with Neural Networks [7.703348666813963]
  We introduce a self-supervised learning approach capable of training on real-world data without a need for ground-truth information. Our neural network is able to run in real-time on both CPU and GPU with suitable sampling rates for autonomous ground robots.
  arXiv  Detail & Related papers  (2021-09-15T08:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.