Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels

• URL: http://arxiv.org/abs/2203.12655v1
• Date: Wed, 23 Mar 2022 18:20:03 GMT
• Title: Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels
• Authors: Bing Li, Cheng Zheng, Guohao Li, Bernard Ghanem
• Abstract summary: We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Our method not only outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground-truth flows.
• Score: 71.11151016581806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Due to the irregularity and sparsity of point clouds, it is expensive and time-consuming to acquire ground-truth scene flow annotations. Some state-of-the-art approaches train scene flow networks in a self-supervised learning manner via approximating pseudo scene flow labels from point clouds. However, these methods fail to achieve the performance level of fully supervised methods, due to the limitations of point cloud such as sparsity and lacking color information. To provide an alternative, we propose a novel approach that utilizes monocular RGB images and point clouds to generate pseudo scene flow labels for training scene flow networks. Our pseudo label generation module infers pseudo scene labels for point clouds by jointly leveraging rich appearance information in monocular images and geometric information of point clouds. To further reduce the negative effect of noisy pseudo labels on the training, we propose a noisy-label-aware training scheme by exploiting the geometric relations of points. Experiment results show that our method not only outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground-truth flows.

Related papers

• Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior [100.98123802027847]
  We investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. We generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation. Our method achieves new state-of-the-art performance in self-supervised scene flow learning.
  arXiv  Detail & Related papers  (2023-10-17T14:06:55Z)
• Data Augmentation-free Unsupervised Learning for 3D Point Cloud Understanding [61.30276576646909]
  We propose an augmentation-free unsupervised approach for point clouds to learn transferable point-level features via soft clustering, named SoftClu. We exploit the affiliation of points to their clusters as a proxy to enable self-training through a pseudo-label prediction task.
  arXiv  Detail & Related papers  (2022-10-06T10:18:16Z)
• 3D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motion [19.419030878019974]
  3D scene flow characterizes how the points at the current time flow to the next time in the 3D Euclidean space. The stability of the predicted scene flow is improved by introducing the dense nature of 2D pixels into the 3D space. Disparity consistency loss is proposed to achieve more effective unsupervised learning of 3D scene flow.
  arXiv  Detail & Related papers  (2022-09-27T03:27:09Z)
• Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance [20.735976558587588]
  Scene flow estimation is used in various applications such as autonomous driving fields, activity recognition, and virtual reality fields. It is challenging to annotate scene flow with ground truth for real-world data. We propose to use odometry information to assist the unsupervised learning of scene flow and use real-world LiDAR data to train our network.
  arXiv  Detail & Related papers  (2022-09-11T21:53:43Z)
• RCP: Recurrent Closest Point for Scene Flow Estimation on 3D Point Clouds [44.034836961967144]
  3D motion estimation including scene flow and point cloud registration has drawn increasing interest. Recent methods employ deep neural networks to construct the cost volume for estimating accurate 3D flow. We decompose the problem into two interlaced stages, where the 3D flows are optimized point-wisely at the first stage and then globally regularized in a recurrent network at the second stage.
  arXiv  Detail & Related papers  (2022-05-23T04:04:30Z)
• Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk [59.87525177207915]
  We develop a self-supervised method to establish correspondences between two point clouds to approximate scene flow. Our method achieves state-of-the-art performance among self-supervised learning methods.
  arXiv  Detail & Related papers  (2021-05-18T03:12:42Z)
• SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation [71.2856098776959]
  Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
  arXiv  Detail & Related papers  (2021-05-10T15:16:14Z)
• Adversarial Self-Supervised Scene Flow Estimation [15.278302535191866]
  This work proposes a metric learning approach for self-supervised scene flow estimation. We outline a benchmark for self-supervised scene flow estimation: the Scene Flow Sandbox.
  arXiv  Detail & Related papers  (2020-11-01T16:37:37Z)
• Scene Flow from Point Clouds with or without Learning [47.03163552693887]
  Scene flow is the three-dimensional (3D) motion field of a scene. Current learning-based approaches seek to estimate the scene flow directly from point clouds. We present a simple and interpretable objective function to recover the scene flow from point clouds.
  arXiv  Detail & Related papers  (2020-10-31T17:24:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.