Picasso: A CUDA-based Library for Deep Learning over 3D Meshes
- URL: http://arxiv.org/abs/2103.15076v1
- Date: Sun, 28 Mar 2021 08:04:50 GMT
- Title: Picasso: A CUDA-based Library for Deep Learning over 3D Meshes
- Authors: Huan Lei, Naveed Akhtar, Ajmal Mian
- Abstract summary: We present Picasso, a library comprising novel modules for deep learning over complex real-world 3D meshes.
We design GPU-accelerated mesh decimation to facilitate network resolution reduction efficiently on-the-fly.
We demonstrate the effectiveness of the proposed modules with competitive segmentation results on S3DIS.
- Score: 46.8917772877766
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present Picasso, a CUDA-based library comprising novel modules for deep
learning over complex real-world 3D meshes. Hierarchical neural architectures
have proved effective in multi-scale feature extraction which signifies the
need for fast mesh decimation. However, existing methods rely on CPU-based
implementations to obtain multi-resolution meshes. We design GPU-accelerated
mesh decimation to facilitate network resolution reduction efficiently
on-the-fly. Pooling and unpooling modules are defined on the vertex clusters
gathered during decimation. For feature learning over meshes, Picasso contains
three types of novel convolutions namely, facet2vertex, vertex2facet, and
facet2facet convolution. Hence, it treats a mesh as a geometric structure
comprising vertices and facets, rather than a spatial graph with edges as
previous methods do. Picasso also incorporates a fuzzy mechanism in its filters
for robustness to mesh sampling (vertex density). It exploits Gaussian mixtures
to define fuzzy coefficients for the facet2vertex convolution, and barycentric
interpolation to define the coefficients for the remaining two convolutions. In
this release, we demonstrate the effectiveness of the proposed modules with
competitive segmentation results on S3DIS. The library will be made public
through https://github.com/hlei-ziyan/Picasso.
Related papers
- MinkUNeXt: Point Cloud-based Large-scale Place Recognition using 3D
Sparse Convolutions [1.124958340749622]
MinkUNeXt is an effective and efficient architecture for place-recognition from point clouds entirely based on the new 3D MinkNeXt Block.
A thorough assessment of the proposal has been carried out using the Oxford RobotCar and the In-house datasets.
arXiv Detail & Related papers (2024-03-12T12:25:54Z) - CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds [55.44204039410225]
We present a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D.
Our proposed method first generates some high-quality 3D proposals by leveraging the class-aware local group strategy on the object surface voxels.
To recover the features of missed voxels due to incorrect voxel-wise segmentation, we build a fully sparse convolutional RoI pooling module.
arXiv Detail & Related papers (2022-10-09T13:38:48Z) - Focal Sparse Convolutional Networks for 3D Object Detection [121.45950754511021]
We introduce two new modules to enhance the capability of Sparse CNNs.
They are focal sparse convolution (Focals Conv) and its multi-modal variant of focal sparse convolution with fusion.
For the first time, we show that spatially learnable sparsity in sparse convolution is essential for sophisticated 3D object detection.
arXiv Detail & Related papers (2022-04-26T17:34:10Z) - Mesh Convolution with Continuous Filters for 3D Surface Parsing [101.25796935464648]
We propose a series of modular operations for effective geometric feature learning from 3D triangle meshes.
Our mesh convolutions exploit spherical harmonics as orthonormal bases to create continuous convolutional filters.
We further contribute a novel hierarchical neural network for perceptual parsing of 3D surfaces, named PicassoNet++.
arXiv Detail & Related papers (2021-12-03T09:16:49Z) - Subdivision-Based Mesh Convolution Networks [38.09613983540932]
Convolutional neural networks (CNNs) have made great breakthroughs in 2D computer vision.
This paper introduces a novel CNN framework, named SubdivNet, for 3D triangle meshes with Loop subdivision sequence connectivity.
Experiments on mesh classification, segmentation, correspondence, and retrieval from the real-world demonstrate the effectiveness and efficiency of SubdivNet.
arXiv Detail & Related papers (2021-06-04T06:50:34Z) - Learning Deformable Tetrahedral Meshes for 3D Reconstruction [78.0514377738632]
3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics.
Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations.
We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
arXiv Detail & Related papers (2020-11-03T02:57:01Z) - Primal-Dual Mesh Convolutional Neural Networks [62.165239866312334]
We propose a primal-dual framework drawn from the graph-neural-network literature to triangle meshes.
Our method takes features for both edges and faces of a 3D mesh as input and dynamically aggregates them.
We provide theoretical insights of our approach using tools from the mesh-simplification literature.
arXiv Detail & Related papers (2020-10-23T14:49:02Z) - DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes [28.571946680616765]
We propose a family of deep hierarchical convolutional networks over 3D geometric data.
The first type, geodesic convolutions, defines the kernel weights over mesh surfaces or graphs.
The second type, Euclidean convolutions, is independent of any underlying mesh structure.
arXiv Detail & Related papers (2020-04-02T13:52:00Z) - Recalibrating 3D ConvNets with Project & Excite [6.11737116137921]
Convolutional Neural Networks (F-CNNs) achieve state-of-the-art performance for segmentation tasks in computer vision and medical imaging.
We extend existing 2D recalibration methods to 3D and propose a generic compress-process-recalibrate pipeline for easy comparison.
We demonstrate that PE modules can be easily integrated into 3D F-CNNs, boosting performance up to 0.3 in Dice Score and outperforming 3D extensions of other recalibration blocks.
arXiv Detail & Related papers (2020-02-25T16:07:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.