Lateral Ego-Vehicle Control without Supervision using Point Clouds

• URL: http://arxiv.org/abs/2203.10662v1
• Date: Sun, 20 Mar 2022 21:57:32 GMT
• Title: Lateral Ego-Vehicle Control without Supervision using Point Clouds
• Authors: Florian M\"uller, Qadeer Khan, Daniel Cremers
• Abstract summary: Existing vision based supervised approaches to lateral vehicle control are capable of directly mapping RGB images to the appropriate steering commands. This paper proposes a framework for training a more robust and scalable model for lateral vehicle control. Online experiments show that the performance of our method is superior to that of the supervised model.
• Score: 50.40632021583213
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing vision based supervised approaches to lateral vehicle control are capable of directly mapping RGB images to the appropriate steering commands. However, they are prone to suffering from inadequate robustness in real world scenarios due to a lack of failure cases in the training data. In this paper, a framework for training a more robust and scalable model for lateral vehicle control is proposed. The framework only requires an unlabeled sequence of RGB images. The trained model takes a point cloud as input and predicts the lateral offset to a subsequent frame from which the steering angle is inferred. The frame poses are in turn obtained from visual odometry. The point cloud is conceived by projecting dense depth maps into 3D. An arbitrary number of additional trajectories from this point cloud can be generated during training. This is to increase the robustness of the model. Online experiments show that the performance of our method is superior to that of the supervised model.

Related papers

• Cross-Modal Self-Supervised Learning with Effective Contrastive Units for LiDAR Point Clouds [34.99995524090838]
  3D perception in LiDAR point clouds is crucial for a self-driving vehicle to properly act in 3D environment. There has been a growing interest in self-supervised pre-training of 3D perception models. We propose the instance-aware and similarity-balanced contrastive units that are tailored for self-driving point clouds.
  arXiv  Detail & Related papers  (2024-09-10T19:11:45Z)
• Adapt PointFormer: 3D Point Cloud Analysis via Adapting 2D Visual Transformers [38.08724410736292]
  This paper attempts to leverage pre-trained models with 2D prior knowledge to accomplish the tasks for 3D point cloud analysis. We propose the Adaptive PointFormer (APF), which fine-tunes pre-trained 2D models with only a modest number of parameters to directly process point clouds.
  arXiv  Detail & Related papers  (2024-07-18T06:32:45Z)
• Point Cloud Models Improve Visual Robustness in Robotic Learners [18.23824531384375]
  We introduce a novel Point Cloud World Model (PCWM) and point cloud based control policies. Our experiments show that policies that explicitly encode point clouds are significantly more robust than their RGB-D counterparts. Taken together, these results suggest reasoning about the 3D scene through point clouds can improve performance, reduce learning time, and increase robustness for robotic learners.
  arXiv  Detail & Related papers  (2024-04-29T17:59:11Z)
• SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior [53.52396082006044]
  Current methods struggle to maintain rendering quality at the viewpoint that deviates significantly from the training viewpoints. This issue stems from the sparse training views captured by a fixed camera on a moving vehicle. We propose a novel approach that enhances the capacity of 3DGS by leveraging prior from a Diffusion Model.
  arXiv  Detail & Related papers  (2024-03-29T09:20:29Z)
• PRED: Pre-training via Semantic Rendering on LiDAR Point Clouds [18.840000859663153]
  We propose PRED, a novel image-assisted pre-training framework for outdoor point clouds. The main ingredient of our framework is a Birds-Eye-View (BEV) feature map conditioned semantic rendering. We further enhance our model's performance by incorporating point-wise masking with a high mask ratio.
  arXiv  Detail & Related papers  (2023-11-08T07:26:09Z)
• Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Models [97.58685709663287]
  generative pre-training can boost the performance of fundamental models in 2D vision. In 3D vision, the over-reliance on Transformer-based backbones and the unordered nature of point clouds have restricted the further development of generative pre-training. We propose a novel 3D-to-2D generative pre-training method that is adaptable to any point cloud model.
  arXiv  Detail & Related papers  (2023-07-27T16:07:03Z)
• Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion [53.93172686610741]
  Cross-PCC is an unsupervised point cloud completion method without requiring any 3D complete point clouds. To take advantage of the complementary information from 2D images, we use a single-view RGB image to extract 2D features. Our method even achieves comparable performance to some supervised methods.
  arXiv  Detail & Related papers  (2022-12-01T15:11:21Z)
• P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting [94.11915008006483]
  We propose a novel Point-to-Pixel prompting for point cloud analysis. Our method attains 89.3% accuracy on the hardest setting of ScanObjectNN. Our framework also exhibits very competitive performance on ModelNet classification and ShapeNet Part Code.
  arXiv  Detail & Related papers  (2022-08-04T17:59:03Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.