3D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motion

• URL: http://arxiv.org/abs/2209.13130v1
• Date: Tue, 27 Sep 2022 03:27:09 GMT
• Title: 3D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motion
• Authors: Chaokang Jiang, Guangming Wang, Yanzi Miao, and Hesheng Wang
• Abstract summary: 3D scene flow characterizes how the points at the current time flow to the next time in the 3D Euclidean space. The stability of the predicted scene flow is improved by introducing the dense nature of 2D pixels into the 3D space. Disparity consistency loss is proposed to achieve more effective unsupervised learning of 3D scene flow.
• Score: 19.419030878019974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D scene flow characterizes how the points at the current time flow to the next time in the 3D Euclidean space, which possesses the capacity to infer autonomously the non-rigid motion of all objects in the scene. The previous methods for estimating scene flow from images have limitations, which split the holistic nature of 3D scene flow by estimating optical flow and disparity separately. Learning 3D scene flow from point clouds also faces the difficulties of the gap between synthesized and real data and the sparsity of LiDAR point clouds. In this paper, the generated dense depth map is utilized to obtain explicit 3D coordinates, which achieves direct learning of 3D scene flow from 2D images. The stability of the predicted scene flow is improved by introducing the dense nature of 2D pixels into the 3D space. Outliers in the generated 3D point cloud are removed by statistical methods to weaken the impact of noisy points on the 3D scene flow estimation task. Disparity consistency loss is proposed to achieve more effective unsupervised learning of 3D scene flow. The proposed method of self-supervised learning of 3D scene flow on real-world images is compared with a variety of methods for learning on the synthesized dataset and learning on LiDAR point clouds. The comparisons of multiple scene flow metrics are shown to demonstrate the effectiveness and superiority of introducing pseudo-LiDAR point cloud to scene flow estimation.

Related papers

• DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds [42.64433313672884]
  We regularize raw points to a dense format by storing 3D coordinates in 2D grids. Unlike the sampling operation commonly used in existing works, the dense 2D representation preserves most points. We also present a novel warping projection technique to alleviate the information loss problem.
  arXiv  Detail & Related papers  (2023-08-08T16:37:24Z)
• Unsupervised Learning of 3D Scene Flow from Monocular Camera [21.34395959441377]
  It is difficult to obtain the ground truth of scene flow in the real scenes, and recent studies are based on synthetic data for training. A novel unsupervised learning method for scene flow is proposed in this paper, which utilizes the images of two consecutive frames taken by monocular camera. Our method realizes the goal that training scene flow network with real-world data, which bridges the gap between training data and test data.
  arXiv  Detail & Related papers  (2022-06-08T04:57:27Z)
• RCP: Recurrent Closest Point for Scene Flow Estimation on 3D Point Clouds [44.034836961967144]
  3D motion estimation including scene flow and point cloud registration has drawn increasing interest. Recent methods employ deep neural networks to construct the cost volume for estimating accurate 3D flow. We decompose the problem into two interlaced stages, where the 3D flows are optimized point-wisely at the first stage and then globally regularized in a recurrent network at the second stage.
  arXiv  Detail & Related papers  (2022-05-23T04:04:30Z)
• Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels [71.11151016581806]
  We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Our method not only outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground-truth flows.
  arXiv  Detail & Related papers  (2022-03-23T18:20:03Z)
• SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation [71.2856098776959]
  Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
  arXiv  Detail & Related papers  (2021-05-10T15:16:14Z)
• FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds [28.899804787744202]
  Scene flow depicts the dynamics of a 3D scene, which is critical for various applications such as autonomous driving, robot navigation, AR/VR, etc. It remains challenging to extract scene flow from point clouds due to sparsity and irregularity in typical point cloud sampling patterns. A novel Spatial Abstraction with Attention (SA2) layer is proposed to alleviate the unstable abstraction problem. A Temporal Abstraction with Attention (TA2) layer is proposed to rectify attention in temporal domain, leading to benefits with motions scaled in a larger range.
  arXiv  Detail & Related papers  (2021-04-01T23:04:04Z)
• How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines [49.06411148698547]
  This paper shows that a significant amount of information about the 3D scene geometry is preserved in line clouds. Our approach is based on the observation that the closest points between lines can yield a good approximation to the original 3D points.
  arXiv  Detail & Related papers  (2021-03-08T21:32:43Z)
• Weakly Supervised Learning of Rigid 3D Scene Flow [81.37165332656612]
  We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. We showcase the effectiveness and generalization capacity of our method on four different autonomous driving datasets.
  arXiv  Detail & Related papers  (2021-02-17T18:58:02Z)
• Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation [87.54570024320354]
  State-of-the-art methods for large-scale driving-scene LiDAR semantic segmentation often project and process the point clouds in the 2D space. A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space. We develop a 3D cylinder partition and a 3D cylinder convolution based framework, termed as Cylinder3D, which exploits the 3D topology relations and structures of driving-scene point clouds.
  arXiv  Detail & Related papers  (2020-08-04T13:56:19Z)
• Self-Supervised Monocular Scene Flow Estimation [27.477810324117016]
  We propose a novel monocular scene flow method that yields competitive accuracy and real-time performance. By taking an inverse problem view, we design a single convolutional neural network (CNN) that successfully estimates depth and 3D motion simultaneously.
  arXiv  Detail & Related papers  (2020-04-08T17:55:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.