Quasi-Balanced Self-Training on Noise-Aware Synthesis of Object Point
Clouds for Closing Domain Gap
- URL: http://arxiv.org/abs/2203.03833v1
- Date: Tue, 8 Mar 2022 03:44:49 GMT
- Title: Quasi-Balanced Self-Training on Noise-Aware Synthesis of Object Point
Clouds for Closing Domain Gap
- Authors: Yongwei Chen, Zihao Wang, Longkun Zou, Ke Chen, Kui Jia
- Abstract summary: We propose an integrated scheme consisting of physically realistic synthesis of object point clouds via rendering stereo images via projection of speckle patterns onto CAD models.
Experiment results can verify the effectiveness of our method as well as both of its modules for unsupervised domain adaptation on point cloud classification.
- Score: 34.590531549797355
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Semantic analyses of object point clouds are largely driven by releasing of
benchmarking datasets, including synthetic ones whose instances are sampled
from object CAD models. However, learning from synthetic data may not
generalize to practical scenarios, where point clouds are typically incomplete,
non-uniformly distributed, and noisy. Such a challenge of Simulation-to-Real
(Sim2Real) domain gap could be mitigated via learning algorithms of domain
adaptation; however, we argue that generation of synthetic point clouds via
more physically realistic rendering is a powerful alternative, as systematic
non-uniform noise patterns can be captured. To this end, we propose an
integrated scheme consisting of physically realistic synthesis of object point
clouds via rendering stereo images via projection of speckle patterns onto CAD
models and a novel quasi-balanced self-training designed for more balanced data
distribution by sparsity-driven selection of pseudo labeled samples for long
tailed classes. Experiment results can verify the effectiveness of our method
as well as both of its modules for unsupervised domain adaptation on point
cloud classification, achieving the state-of-the-art performance.
Related papers
- PU-Ray: Domain-Independent Point Cloud Upsampling via Ray Marching on Neural Implicit Surface [5.78575346449322]
We propose a new ray-based upsampling approach with an arbitrary rate, where a depth prediction is made for each query ray and its corresponding patch.
Our novel method simulates the sphere-tracing ray marching algorithm on the neural implicit surface defined with an unsigned distance function (UDF)
The rule-based mid-point query sampling method generates more evenly distributed points without requiring an end-to-end model trained using a nearest-neighbor-based reconstruction loss function.
arXiv Detail & Related papers (2023-10-12T22:45:03Z) - Compositional Semantic Mix for Domain Adaptation in Point Cloud
Segmentation [65.78246406460305]
compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation.
We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
arXiv Detail & Related papers (2023-08-28T14:43:36Z) - Exploring the Effectiveness of Dataset Synthesis: An application of
Apple Detection in Orchards [68.95806641664713]
We explore the usability of Stable Diffusion 2.1-base for generating synthetic datasets of apple trees for object detection.
We train a YOLOv5m object detection model to predict apples in a real-world apple detection dataset.
Results demonstrate that the model trained on generated data is slightly underperforming compared to a baseline model trained on real-world images.
arXiv Detail & Related papers (2023-06-20T09:46:01Z) - Controllable Mesh Generation Through Sparse Latent Point Diffusion
Models [105.83595545314334]
We design a novel sparse latent point diffusion model for mesh generation.
Our key insight is to regard point clouds as an intermediate representation of meshes, and model the distribution of point clouds instead.
Our proposed sparse latent point diffusion model achieves superior performance in terms of generation quality and controllability.
arXiv Detail & Related papers (2023-03-14T14:25:29Z) - Synthetic-to-Real Domain Generalized Semantic Segmentation for 3D Indoor
Point Clouds [69.64240235315864]
This paper introduces the synthetic-to-real domain generalization setting to this task.
The domain gap between synthetic and real-world point cloud data mainly lies in the different layouts and point patterns.
Experiments on the synthetic-to-real benchmark demonstrate that both CINMix and multi-prototypes can narrow the distribution gap.
arXiv Detail & Related papers (2022-12-09T05:07:43Z) - Energy-Based Residual Latent Transport for Unsupervised Point Cloud
Completion [39.102570189978934]
Unsupervised point cloud completion aims to infer the whole geometry of a partial object observation without requiring partial-complete correspondence.
We propose a novel framework that performs completion by transforming a partial shape encoding into a complete one using a latent transport module.
We experimentally show that the proposed method produces high-fidelity completion results, outperforming state-of-the-art models by a significant margin.
arXiv Detail & Related papers (2022-11-13T05:16:43Z) - DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation
and Rendering of Deformable Objects [18.266002992029716]
Research in manipulation of deformable objects is typically conducted on a limited range of scenarios.
Realistic simulators with support for various types of deformations and interactions have the potential to speed up experimentation.
For highly deformable objects it is challenging to align the output of a simulator with the behavior of real objects.
arXiv Detail & Related papers (2022-04-07T00:45:26Z) - CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE)
At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales.
We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
arXiv Detail & Related papers (2022-03-03T05:58:49Z) - Quantifying point cloud realism through adversarially learned latent
representations [0.38233569758620056]
This paper presents a novel approach to quantify the realism of local regions in LiDAR point clouds.
The resulting metric can assign a quality score to samples without requiring any task specific annotations.
As one important application, we demonstrate how the local realism score can be used for anomaly detection in point clouds.
arXiv Detail & Related papers (2021-09-24T07:17:27Z) - SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine
Reconstruction with Self-Projection Optimization [52.20602782690776]
It is expensive and tedious to obtain large scale paired sparse-canned point sets for training from real scanned sparse data.
We propose a self-supervised point cloud upsampling network, named SPU-Net, to capture the inherent upsampling patterns of points lying on the underlying object surface.
We conduct various experiments on both synthetic and real-scanned datasets, and the results demonstrate that we achieve comparable performance to the state-of-the-art supervised methods.
arXiv Detail & Related papers (2020-12-08T14:14:09Z) - Style-transfer GANs for bridging the domain gap in synthetic pose
estimator training [8.508403388002133]
We propose to adopt general-purpose GAN models for pixel-level image translation.
The obtained models are then used either during training or inference to bridge the domain gap.
Our evaluation shows a considerable improvement in model performance when compared to a model trained with the same degree of domain randomization.
arXiv Detail & Related papers (2020-04-28T17:35:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.