Mesh Convolution with Continuous Filters for 3D Surface Parsing

• URL: http://arxiv.org/abs/2112.01801v3
• Date: Sat, 22 Apr 2023 02:14:33 GMT
• Title: Mesh Convolution with Continuous Filters for 3D Surface Parsing
• Authors: Huan Lei, Naveed Akhtar, Mubarak Shah, and Ajmal Mian
• Abstract summary: We propose a series of modular operations for effective geometric feature learning from 3D triangle meshes. Our mesh convolutions exploit spherical harmonics as orthonormal bases to create continuous convolutional filters. We further contribute a novel hierarchical neural network for perceptual parsing of 3D surfaces, named PicassoNet++.
• Score: 101.25796935464648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geometric feature learning for 3D surfaces is critical for many applications in computer graphics and 3D vision. However, deep learning currently lags in hierarchical modeling of 3D surfaces due to the lack of required operations and/or their efficient implementations. In this paper, we propose a series of modular operations for effective geometric feature learning from 3D triangle meshes. These operations include novel mesh convolutions, efficient mesh decimation and associated mesh (un)poolings. Our mesh convolutions exploit spherical harmonics as orthonormal bases to create continuous convolutional filters. The mesh decimation module is GPU-accelerated and able to process batched meshes on-the-fly, while the (un)pooling operations compute features for up/down-sampled meshes. We provide open-source implementation of these operations, collectively termed Picasso. Picasso supports heterogeneous mesh batching and processing. Leveraging its modular operations, we further contribute a novel hierarchical neural network for perceptual parsing of 3D surfaces, named PicassoNet++. It achieves highly competitive performance for shape analysis and scene segmentation on prominent 3D benchmarks. The code, data and trained models are available at https://github.com/EnyaHermite/Picasso.

Related papers

• Bridging 3D Gaussian and Mesh for Freeview Video Rendering [57.21847030980905]
  GauMesh bridges the 3D Gaussian and Mesh for modeling and rendering the dynamic scenes. We show that our approach adapts the appropriate type of primitives to represent the different parts of the dynamic scene.
  arXiv  Detail & Related papers  (2024-03-18T04:01:26Z)
• SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation [53.83313235792596]
  We present a new methodology for real-time semantic mapping from RGB-D sequences. It combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping. Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems.
  arXiv  Detail & Related papers  (2023-06-28T22:36:44Z)
• Smooth Mesh Estimation from Depth Data using Non-Smooth Convex Optimization [28.786685021545622]
  We build a 3D mesh directly from a depth map and the sparse landmarks triangulated with visual odometry. Our approach generates a smooth and accurate 3D mesh that substantially improves the state-of-the-art on direct mesh reconstruction while running in real-time.
  arXiv  Detail & Related papers  (2021-08-06T06:29:34Z)
• Subdivision-Based Mesh Convolution Networks [38.09613983540932]
  Convolutional neural networks (CNNs) have made great breakthroughs in 2D computer vision. This paper introduces a novel CNN framework, named SubdivNet, for 3D triangle meshes with Loop subdivision sequence connectivity. Experiments on mesh classification, segmentation, correspondence, and retrieval from the real-world demonstrate the effectiveness and efficiency of SubdivNet.
  arXiv  Detail & Related papers  (2021-06-04T06:50:34Z)
• Picasso: A CUDA-based Library for Deep Learning over 3D Meshes [46.8917772877766]
  We present Picasso, a library comprising novel modules for deep learning over complex real-world 3D meshes. We design GPU-accelerated mesh decimation to facilitate network resolution reduction efficiently on-the-fly. We demonstrate the effectiveness of the proposed modules with competitive segmentation results on S3DIS.
  arXiv  Detail & Related papers  (2021-03-28T08:04:50Z)
• Deep Active Surface Models [60.027353171412216]
  Active Surface Models have a long history of being useful to model complex 3D surfaces but only Active Contours have been used in conjunction with deep networks. We introduce layers that implement them that can be integrated seamlessly into Graph Convolutional Networks to enforce sophisticated smoothness priors.
  arXiv  Detail & Related papers  (2020-11-17T18:48:28Z)
• Making a Case for 3D Convolutions for Object Segmentation in Videos [16.167397418720483]
  We show that 3D convolutional networks can be effectively applied to dense video prediction tasks such as salient object segmentation. We propose a 3D decoder architecture, that comprises novel 3D Global Convolution layers and 3D Refinement modules. Our approach outperforms existing state-of-the-arts by a large margin on the DAVIS'16 Unsupervised, FBMS and ViSal benchmarks.
  arXiv  Detail & Related papers  (2020-08-26T12:24:23Z)
• Learning Local Neighboring Structure for Robust 3D Shape Representation [143.15904669246697]
  Representation learning for 3D meshes is important in many computer vision and graphics applications. We propose a local structure-aware anisotropic convolutional operation (LSA-Conv) Our model produces significant improvement in 3D shape reconstruction compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-04-21T13:40:03Z)
• DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes [28.571946680616765]
  We propose a family of deep hierarchical convolutional networks over 3D geometric data. The first type, geodesic convolutions, defines the kernel weights over mesh surfaces or graphs. The second type, Euclidean convolutions, is independent of any underlying mesh structure.
  arXiv  Detail & Related papers  (2020-04-02T13:52:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.