OmniColor: A Global Camera Pose Optimization Approach of LiDAR-360Camera Fusion for Colorizing Point Clouds

• URL: http://arxiv.org/abs/2404.04693v2
• Date: Thu, 26 Sep 2024 13:53:33 GMT
• Title: OmniColor: A Global Camera Pose Optimization Approach of LiDAR-360Camera Fusion for Colorizing Point Clouds
• Authors: Bonan Liu, Guoyang Zhao, Jianhao Jiao, Guang Cai, Chengyang Li, Handi Yin, Yuyang Wang, Ming Liu, Pan Hui,
• Abstract summary: A Colored point cloud, as a simple and efficient 3D representation, has many advantages in various fields. This paper presents OmniColor, a novel and efficient algorithm to colorize point clouds using an independent 360-degree camera.
• Score: 15.11376768491973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A Colored point cloud, as a simple and efficient 3D representation, has many advantages in various fields, including robotic navigation and scene reconstruction. This representation is now commonly used in 3D reconstruction tasks relying on cameras and LiDARs. However, fusing data from these two types of sensors is poorly performed in many existing frameworks, leading to unsatisfactory mapping results, mainly due to inaccurate camera poses. This paper presents OmniColor, a novel and efficient algorithm to colorize point clouds using an independent 360-degree camera. Given a LiDAR-based point cloud and a sequence of panorama images with initial coarse camera poses, our objective is to jointly optimize the poses of all frames for mapping images onto geometric reconstructions. Our pipeline works in an off-the-shelf manner that does not require any feature extraction or matching process. Instead, we find optimal poses by directly maximizing the photometric consistency of LiDAR maps. In experiments, we show that our method can overcome the severe visual distortion of omnidirectional images and greatly benefit from the wide field of view (FOV) of 360-degree cameras to reconstruct various scenarios with accuracy and stability. The code will be released at https://github.com/liubonan123/OmniColor/.

Related papers

• SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild [76.21063993398451]
  Inverse rendering of an object based on unconstrained image collections is a long-standing challenge in computer vision and graphics. We show that an implicit shape representation based on a multi-resolution hash encoding enables faster and robust shape reconstruction. Our method is class-agnostic and works on in-the-wild image collections of objects to produce relightable 3D assets.
  arXiv  Detail & Related papers  (2024-01-18T18:01:19Z)
• PERF: Panoramic Neural Radiance Field from a Single Panorama [109.31072618058043]
  PERF is a novel view synthesis framework that trains a panoramic neural radiance field from a single panorama. We propose a novel collaborative RGBD inpainting method and a progressive inpainting-and-erasing method to lift up a 360-degree 2D scene to a 3D scene. Our PERF can be widely used for real-world applications, such as panorama-to-3D, text-to-3D, and 3D scene stylization applications.
  arXiv  Detail & Related papers  (2023-10-25T17:59:01Z)
• TriVol: Point Cloud Rendering via Triple Volumes [57.305748806545026]
  We present a dense while lightweight 3D representation, named TriVol, that can be combined with NeRF to render photo-realistic images from point clouds. Our framework has excellent generalization ability to render a category of scenes/objects without fine-tuning.
  arXiv  Detail & Related papers  (2023-03-29T06:34:12Z)
• Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields [63.21420081888606]
  Recent Radiance Fields and extensions are proposed to synthesize realistic images from 2D input. We present Point2Pix as a novel point to link the 3D sparse point clouds with 2D dense image pixels.
  arXiv  Detail & Related papers  (2023-03-29T06:26:55Z)
• Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion [53.93172686610741]
  Cross-PCC is an unsupervised point cloud completion method without requiring any 3D complete point clouds. To take advantage of the complementary information from 2D images, we use a single-view RGB image to extract 2D features. Our method even achieves comparable performance to some supervised methods.
  arXiv  Detail & Related papers  (2022-12-01T15:11:21Z)
• Large-Scale 3D Semantic Reconstruction for Automated Driving Vehicles with Adaptive Truncated Signed Distance Function [9.414880946870916]
  We propose a novel 3D reconstruction and semantic mapping system using LiDAR and camera sensors. An Adaptive Truncated Function is introduced to describe surfaces implicitly, which can deal with different LiDAR point sparsities. An optimal image patch selection strategy is proposed to estimate the optimal semantic class for each triangle mesh.
  arXiv  Detail & Related papers  (2022-02-28T15:11:25Z)
• ADOP: Approximate Differentiable One-Pixel Point Rendering [7.69748487650283]
  We present a point-based, differentiable neural rendering pipeline for scene refinement and novel view synthesis. We show that our system is able to synthesize sharper and more consistent novel views than existing approaches.
  arXiv  Detail & Related papers  (2021-10-13T10:55:39Z)
• PICCOLO: Point Cloud-Centric Omnidirectional Localization [20.567452635590943]
  We present PICCOLO, a simple and efficient algorithm for omnidirectional localization. Our pipeline works in an off-the-shelf manner with a single image given as a query. PICCOLO outperforms existing omnidirectional localization algorithms in both accuracy and stability when evaluated in various environments.
  arXiv  Detail & Related papers  (2021-08-14T14:19:37Z)
• Real-time dense 3D Reconstruction from monocular video data captured by low-cost UAVs [0.3867363075280543]
  Real-time 3D reconstruction enables fast dense mapping of the environment which benefits numerous applications, such as navigation or live evaluation of an emergency. In contrast to most real-time capable approaches, our approach does not need an explicit depth sensor. By exploiting the self-motion of the unmanned aerial vehicle (UAV) flying with oblique view around buildings, we estimate both camera trajectory and depth for selected images with enough novel content.
  arXiv  Detail & Related papers  (2021-04-21T13:12:17Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)
• Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras [13.24490469380487]
  We present an effective multi-view approach to end-to-end learning of precise manipulation tasks that are 3D in nature. Our method learns to accomplish these tasks using multiple statically placed but uncalibrated RGB camera views without building an explicit 3D representation such as a pointcloud or voxel grid.
  arXiv  Detail & Related papers  (2020-02-21T03:28:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.