3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling

• URL: http://arxiv.org/abs/2402.18146v2
• Date: Fri, 1 Mar 2024 04:26:59 GMT
• Title: 3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling
• Authors: Chaokang Jiang, Guangming Wang, Jiuming Liu, Hesheng Wang, Zhuang Ma, Zhenqiang Liu, Zhujin Liang, Yi Shan, Dalong Du
• Abstract summary: We present a novel approach to generate a large number of 3D scene flow pseudo labels for real-world LiDAR point clouds. Specifically, we employ the assumption of rigid body motion to simulate potential object-level rigid movements in autonomous driving scenarios. By perfectly synthesizing target point clouds based on augmented motion parameters, we easily obtain lots of 3D scene flow labels in point clouds highly consistent with real scenarios.
• Score: 21.726386822643995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning 3D scene flow from LiDAR point clouds presents significant difficulties, including poor generalization from synthetic datasets to real scenes, scarcity of real-world 3D labels, and poor performance on real sparse LiDAR point clouds. We present a novel approach from the perspective of auto-labelling, aiming to generate a large number of 3D scene flow pseudo labels for real-world LiDAR point clouds. Specifically, we employ the assumption of rigid body motion to simulate potential object-level rigid movements in autonomous driving scenarios. By updating different motion attributes for multiple anchor boxes, the rigid motion decomposition is obtained for the whole scene. Furthermore, we developed a novel 3D scene flow data augmentation method for global and local motion. By perfectly synthesizing target point clouds based on augmented motion parameters, we easily obtain lots of 3D scene flow labels in point clouds highly consistent with real scenarios. On multiple real-world datasets including LiDAR KITTI, nuScenes, and Argoverse, our method outperforms all previous supervised and unsupervised methods without requiring manual labelling. Impressively, our method achieves a tenfold reduction in EPE3D metric on the LiDAR KITTI dataset, reducing it from $0.190m$ to a mere $0.008m$ error.

Related papers

• Precise Workcell Sketching from Point Clouds Using an AR Toolbox [1.249418440326334]
  Capturing real-world 3D spaces as point clouds is efficient and descriptive, but it comes with sensor errors and lacks object parametrization. Our method for 3D workcell sketching from point clouds allows users to refine raw point clouds using an Augmented Reality interface. By utilizing a toolbox and an AR-enabled pointing device, users can enhance point cloud accuracy based on the device's position in 3D space.
  arXiv  Detail & Related papers  (2024-10-01T08:07:51Z)
• 3DHacker: Spectrum-based Decision Boundary Generation for Hard-label 3D Point Cloud Attack [64.83391236611409]
  We propose a novel 3D attack method to generate adversarial samples solely with the knowledge of class labels. Even in the challenging hard-label setting, 3DHacker still competitively outperforms existing 3D attacks regarding the attack performance as well as adversary quality.
  arXiv  Detail & Related papers  (2023-08-15T03:29:31Z)
• An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds [50.19288542498838]
  3D single object tracking in LiDAR point clouds (LiDAR SOT) plays a crucial role in autonomous driving. Current approaches all follow the Siamese paradigm based on appearance matching. We introduce a motion-centric paradigm to handle LiDAR SOT from a new perspective.
  arXiv  Detail & Related papers  (2023-03-21T17:28:44Z)
• SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving [98.74706005223685]
  3D scene understanding plays a vital role in vision-based autonomous driving. We propose a SurroundOcc method to predict the 3D occupancy with multi-camera images.
  arXiv  Detail & Related papers  (2023-03-16T17:59:08Z)
• GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flow [3.2738068278607426]
  We propose a GMA3D module based on the transformer framework to infer the motion information of occluded points from the motion information of local and global non-occluded points respectively. Experiments show that our GMA3D can solve the occlusion problem in the scene flow, especially in the real scene.
  arXiv  Detail & Related papers  (2022-10-07T03:09:00Z)
• 3D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motion [19.419030878019974]
  3D scene flow characterizes how the points at the current time flow to the next time in the 3D Euclidean space. The stability of the predicted scene flow is improved by introducing the dense nature of 2D pixels into the 3D space. Disparity consistency loss is proposed to achieve more effective unsupervised learning of 3D scene flow.
  arXiv  Detail & Related papers  (2022-09-27T03:27:09Z)
• PolarMix: A General Data Augmentation Technique for LiDAR Point Clouds [100.03877236181546]
  PolarMix is a point cloud augmentation technique that is simple and generic. It can work as plug-and-play for various 3D deep architectures and also performs well for unsupervised domain adaptation.
  arXiv  Detail & Related papers  (2022-07-30T13:52:19Z)
• Dynamic 3D Scene Analysis by Point Cloud Accumulation [32.491921765128936]
  Multi-beam LiDAR sensors are used on autonomous vehicles and mobile robots. Each frame covers the scene sparsely, due to limited angular scanning resolution and occlusion. We propose a method that exploits inductive biases of outdoor street scenes, including their geometric layout and object-level rigidity.
  arXiv  Detail & Related papers  (2022-07-25T17:57:46Z)
• Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels [71.11151016581806]
  We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Our method not only outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground-truth flows.
  arXiv  Detail & Related papers  (2022-03-23T18:20:03Z)
• Scalable Scene Flow from Point Clouds in the Real World [30.437100097997245]
  We introduce a new large scale benchmark for scene flow based on the Open dataset. We show how previous works were bounded based on the amount of real LiDAR data available. We introduce the model architecture FastFlow3D that provides real time inference on the full point cloud.
  arXiv  Detail & Related papers  (2021-03-01T20:56:05Z)
• Weakly Supervised Learning of Rigid 3D Scene Flow [81.37165332656612]
  We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. We showcase the effectiveness and generalization capacity of our method on four different autonomous driving datasets.
  arXiv  Detail & Related papers  (2021-02-17T18:58:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.