PaintNet: Unstructured Multi-Path Learning from 3D Point Clouds for Robotic Spray Painting

• URL: http://arxiv.org/abs/2211.06930v3
• Date: Wed, 6 Dec 2023 14:59:53 GMT
• Title: PaintNet: Unstructured Multi-Path Learning from 3D Point Clouds for Robotic Spray Painting
• Authors: Gabriele Tiboni, Raffaello Camoriano, Tatiana Tommasi
• Abstract summary: Industrial robotic problems such as spray painting and welding require planning of multiple trajectories to solve the task. Existing solutions make strong assumptions on the form of input surfaces and the nature of output paths. By leveraging on recent advances in 3D deep learning, we introduce a novel framework capable of dealing with arbitrary 3D surfaces.
• Score: 13.182797149468204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Popular industrial robotic problems such as spray painting and welding require (i) conditioning on free-shape 3D objects and (ii) planning of multiple trajectories to solve the task. Yet, existing solutions make strong assumptions on the form of input surfaces and the nature of output paths, resulting in limited approaches unable to cope with real-data variability. By leveraging on recent advances in 3D deep learning, we introduce a novel framework capable of dealing with arbitrary 3D surfaces, and handling a variable number of unordered output paths (i.e. unstructured). Our approach predicts local path segments, which can be later concatenated to reconstruct long-horizon paths. We extensively validate the proposed method in the context of robotic spray painting by releasing PaintNet, the first public dataset of expert demonstrations on free-shape 3D objects collected in a real industrial scenario. A thorough experimental analysis demonstrates the capabilities of our model to promptly predict smooth output paths that cover up to 95% of previously unseen object surfaces, even without explicitly optimizing for paint coverage.

Related papers

• OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments [77.0399450848749]
  We propose an OccNeRF method for training occupancy networks without 3D supervision. We parameterize the reconstructed occupancy fields and reorganize the sampling strategy to align with the cameras' infinite perceptive range. For semantic occupancy prediction, we design several strategies to polish the prompts and filter the outputs of a pretrained open-vocabulary 2D segmentation model.
  arXiv  Detail & Related papers  (2023-12-14T18:58:52Z)
• Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priors [15.34487368683311]
  We propose a framework that can reconstruct high-quality object-level maps for unknown objects. Our approach takes multiple RGB-D images as input and outputs dense 3D shapes and 9-DoF poses for detected objects. We derive a probabilistic formulation that propagates shape and pose uncertainty through two novel loss functions.
  arXiv  Detail & Related papers  (2023-09-17T00:48:19Z)
• Free-form 3D Scene Inpainting with Dual-stream GAN [20.186778638697696]
  We present a novel task named free-form 3D scene inpainting. Unlike scenes in previous 3D completion datasets, the proposed inpainting dataset contains large and diverse missing regions. Our dual-stream generator, fusing both geometry and color information, produces distinct semantic boundaries. To further enhance the details, our lightweight dual-stream discriminator regularizes the geometry and color edges of the predicted scenes to be realistic and sharp.
  arXiv  Detail & Related papers  (2022-12-16T13:20:31Z)
• Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion [54.151979979158085]
  We introduce a principled end-to-end reconstruction framework for natural images, where accurate ground-truth poses are not available. We leverage an unconditional 3D-aware generator, to which we apply a hybrid inversion scheme where a model produces a first guess of the solution. Our framework can de-render an image in as few as 10 steps, enabling its use in practical scenarios.
  arXiv  Detail & Related papers  (2022-11-21T17:42:42Z)
• Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields [82.21033337949757]
  This paper introduces a ray-based volumetric uncertainty estimator, which computes the entropy of the weight distribution of the color samples along each ray of the object's implicit neural representation. We show that it is possible to infer the uncertainty of the underlying 3D geometry given a novel view with the proposed estimator. We present a next-best-view selection policy guided by the ray-based volumetric uncertainty in neural radiance fields-based representations.
  arXiv  Detail & Related papers  (2022-09-17T21:28:57Z)
• Towards Confidence-guided Shape Completion for Robotic Applications [6.940242990198]
  Deep learning has begun taking traction as effective means of inferring a complete 3D object representation from partial visual data. We propose an object shape completion method based on an implicit 3D representation providing a confidence value for each reconstructed point. We experimentally validate our approach by comparing reconstructed shapes with ground truths, and by deploying our shape completion algorithm in a robotic grasping pipeline.
  arXiv  Detail & Related papers  (2022-09-09T13:48:24Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)
• DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes [54.239416488865565]
  We propose a fast single-stage 3D object detection method for LIDAR data. The core novelty of our method is a fast, single-pass architecture that both detects objects in 3D and estimates their shapes. We find that our proposed method achieves state-of-the-art results by 5% on object detection in ScanNet scenes, and it gets top results by 3.4% in the Open dataset.
  arXiv  Detail & Related papers  (2020-04-02T17:48:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.