Object Dynamics Modeling with Hierarchical Point Cloud-based Representations

• URL: http://arxiv.org/abs/2404.06044v1
• Date: Tue, 9 Apr 2024 06:10:15 GMT
• Title: Object Dynamics Modeling with Hierarchical Point Cloud-based Representations
• Authors: Chanho Kim, Li Fuxin,
• Abstract summary: We propose a novel U-net architecture based on continuous point convolution which embeds information from 3D coordinates. Bottleneck layers in the downsampled point clouds lead to better long-range interaction modeling. Our approach significantly improves the state-of-the-art, especially in scenarios that require accurate gravity or collision reasoning.
• Score: 1.3934784414106087
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modeling object dynamics with a neural network is an important problem with numerous applications. Most recent work has been based on graph neural networks. However, physics happens in 3D space, where geometric information potentially plays an important role in modeling physical phenomena. In this work, we propose a novel U-net architecture based on continuous point convolution which naturally embeds information from 3D coordinates and allows for multi-scale feature representations with established downsampling and upsampling procedures. Bottleneck layers in the downsampled point clouds lead to better long-range interaction modeling. Besides, the flexibility of point convolutions allows our approach to generalize to sparsely sampled points from mesh vertices and dynamically generate features on important interaction points on mesh faces. Experimental results demonstrate that our approach significantly improves the state-of-the-art, especially in scenarios that require accurate gravity or collision reasoning.

Related papers

• Geometry Distributions [51.4061133324376]
  We propose a novel geometric data representation that models geometry as distributions. Our approach uses diffusion models with a novel network architecture to learn surface point distributions. We evaluate our representation qualitatively and quantitatively across various object types, demonstrating its effectiveness in achieving high geometric fidelity.
  arXiv  Detail & Related papers  (2024-11-25T04:06:48Z)
• Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories [28.701879490459675]
  We aim to learn an implicit motion field parameterized by a neural network to predict the movement of novel points within same domain. We exploit intrinsic regularization provided by SIREN, and modify the input layer to produce atemporally smooth motion field. Our experiments assess the model's performance in predicting unseen point trajectories and its application in temporal mesh alignment with deformation.
  arXiv  Detail & Related papers  (2024-06-05T21:02:10Z)
• ParaPoint: Learning Global Free-Boundary Surface Parameterization of 3D Point Clouds [52.03819676074455]
  ParaPoint is an unsupervised neural learning pipeline for achieving global free-boundary surface parameterization. This work makes the first attempt to investigate neural point cloud parameterization that pursues both global mappings and free boundaries.
  arXiv  Detail & Related papers  (2024-03-15T14:35:05Z)
• AGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects [7.414594429329531]
  We propose an improved architecture for point cloud prediction of deformable 3D objects. Specifically, to handle deformable shapes, we propose a graph-based approach that learns and exploits the spatial structure of point clouds. The proposed adaptative module controls the composition of local and global motions for each point, enabling the network to model complex motions in deformable 3D objects more effectively.
  arXiv  Detail & Related papers  (2023-07-19T12:21:39Z)
• Dynamic Point Fields [30.029872787758705]
  We present a dynamic point field model that combines the representational benefits of explicit point-based graphics with implicit deformation networks. We show the advantages of our dynamic point field framework in terms of its representational power, learning efficiency, and robustness to out-of-distribution novel poses.
  arXiv  Detail & Related papers  (2023-04-05T17:52:37Z)
• StarNet: Style-Aware 3D Point Cloud Generation [82.30389817015877]
  StarNet is able to reconstruct and generate high-fidelity and even 3D point clouds using a mapping network. Our framework achieves comparable state-of-the-art performance on various metrics in the point cloud reconstruction and generation tasks.
  arXiv  Detail & Related papers  (2023-03-28T08:21:44Z)
• Learning Multi-Object Dynamics with Compositional Neural Radiance Fields [63.424469458529906]
  We present a method to learn compositional predictive models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks. NeRFs have become a popular choice for representing scenes due to their strong 3D prior. For planning, we utilize RRTs in the learned latent space, where we can exploit our model and the implicit object encoder to make sampling the latent space informative and more efficient.
  arXiv  Detail & Related papers  (2022-02-24T01:31:29Z)
• Concentric Spherical GNN for 3D Representation Learning [53.45704095146161]
  We propose a novel multi-resolution convolutional architecture for learning over concentric spherical feature maps. Our hierarchical architecture is based on alternatively learning to incorporate both intra-sphere and inter-sphere information. We demonstrate the effectiveness of our approach in improving state-of-the-art performance on 3D classification tasks with rotated data.
  arXiv  Detail & Related papers  (2021-03-18T19:05:04Z)
• Local Grid Rendering Networks for 3D Object Detection in Point Clouds [98.02655863113154]
  CNNs are powerful but it would be computationally costly to directly apply convolutions on point data after voxelizing the entire point clouds to a dense regular 3D grid. We propose a novel and principled Local Grid Rendering (LGR) operation to render the small neighborhood of a subset of input points into a low-resolution 3D grid independently. We validate LGR-Net for 3D object detection on the challenging ScanNet and SUN RGB-D datasets.
  arXiv  Detail & Related papers  (2020-07-04T13:57:43Z)
• HyperFlow: Representing 3D Objects as Surfaces [19.980044265074298]
  We present a novel generative model that leverages hypernetworks to create continuous 3D object representations in a form of lightweight surfaces (meshes) directly out of point clouds. We obtain continuous mesh-based object representations that yield better qualitative results than competing approaches.
  arXiv  Detail & Related papers  (2020-06-15T19:18:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.