PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

• URL: http://arxiv.org/abs/2208.03720v1
• Date: Sun, 7 Aug 2022 13:37:29 GMT
• Title: PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs
• Authors: Zhengyang Shen, Tao Hong, Qi She, Jinwen Ma, Zhouchen Lin
• Abstract summary: In this work, we employ partial differential operators (PDOs) to model 3D filters, and derive general steerable 3D CNNs called PDO-s3DCNNs. We prove that the equivariant filters are subject to linear constraints, which can be solved efficiently under various conditions.
• Score: 69.85869748832127
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Steerable models can provide very general and flexible equivariance by formulating equivariance requirements in the language of representation theory and feature fields, which has been recognized to be effective for many vision tasks. However, deriving steerable models for 3D rotations is much more difficult than that in the 2D case, due to more complicated mathematics of 3D rotations. In this work, we employ partial differential operators (PDOs) to model 3D filters, and derive general steerable 3D CNNs, which are called PDO-s3DCNNs. We prove that the equivariant filters are subject to linear constraints, which can be solved efficiently under various conditions. As far as we know, PDO-s3DCNNs are the most general steerable CNNs for 3D rotations, in the sense that they cover all common subgroups of $SO(3)$ and their representations, while existing methods can only be applied to specific groups and representations. Extensive experiments show that our models can preserve equivariance well in the discrete domain, and outperform previous works on SHREC'17 retrieval and ISBI 2012 segmentation tasks with a low network complexity.

Related papers

• 3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction [50.07071392673984]
  Existing methods learn 3D rotations parametrized in the spatial domain using angles or quaternions. We propose a frequency-domain approach that directly predicts Wigner-D coefficients for 3D rotation regression. Our method achieves state-of-the-art results on benchmarks such as ModelNet10-SO(3) and PASCAL3D+.
  arXiv  Detail & Related papers  (2024-11-01T12:50:38Z)
• Exploiting the Complementarity of 2D and 3D Networks to Address Domain-Shift in 3D Semantic Segmentation [14.30113021974841]
  3D semantic segmentation is a critical task in many real-world applications, such as autonomous driving, robotics, and mixed reality. A possible solution is to combine the 3D information with others coming from sensors featuring a different modality, such as RGB cameras. Recent multi-modal 3D semantic segmentation networks exploit these modalities relying on two branches that process the 2D and 3D information independently.
  arXiv  Detail & Related papers  (2023-04-06T10:59:43Z)
• SVNet: Where SO(3) Equivariance Meets Binarization on Point Cloud Representation [65.4396959244269]
  The paper tackles the challenge by designing a general framework to construct 3D learning architectures. The proposed approach can be applied to general backbones like PointNet and DGCNN. Experiments on ModelNet40, ShapeNet, and the real-world dataset ScanObjectNN, demonstrated that the method achieves a great trade-off between efficiency, rotation, and accuracy.
  arXiv  Detail & Related papers  (2022-09-13T12:12:19Z)
• Group Shift Pointwise Convolution for Volumetric Medical Image Segmentation [31.72090839643412]
  We introduce a novel Group Shift Pointwise Convolution (GSP-Conv) to improve the effectiveness and efficiency of 3D convolutions. GSP-Conv simplifies 3D convolutions into pointwise ones with 1x1x1 kernels, which dramatically reduces the number of model parameters and FLOPs. Results show that our method, with substantially decreased model complexity, achieves comparable or even better performance than models employing 3D convolutions.
  arXiv  Detail & Related papers  (2021-09-26T15:27:33Z)
• Asymmetric 3D Context Fusion for Universal Lesion Detection [55.61873234187917]
  3D networks are strong in 3D context yet lack supervised pretraining. Existing 3D context fusion operators are designed to be spatially symmetric, performing identical operations on each 2D slice like convolutions. We propose a novel asymmetric 3D context fusion operator (A3D), which uses different weights to fuse 3D context from different 2D slices.
  arXiv  Detail & Related papers  (2021-09-17T16:25:10Z)
• PDO-e$\text{S}^\text{2}$CNNs: Partial Differential Operator Based Equivariant Spherical CNNs [77.53203546732664]
  We use partial differential operators to design a spherical equivariant CNN, PDO-e$textStext2$CNN, which is exactly rotation equivariant in the continuous domain. In experiments, PDO-e$textStext2$CNNs show greater parameter efficiency and outperform other spherical CNNs significantly on several tasks.
  arXiv  Detail & Related papers  (2021-04-08T07:54:50Z)
• Equivariant Point Network for 3D Point Cloud Analysis [17.689949017410836]
  We propose an effective and practical SE(3) (3D translation and rotation) equivariant network for point cloud analysis. First, we present SE(3) separable point convolution, a novel framework that breaks down the 6D convolution into two separable convolutional operators. Second, we introduce an attention layer to effectively harness the expressiveness of the equivariant features.
  arXiv  Detail & Related papers  (2021-03-25T21:57:10Z)
• Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes [86.2129580231191]
  Adjoint Rigid Transform (ART) Network is a neural module which can be integrated with a variety of 3D networks. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks. We will release our code and pre-trained models for further research.
  arXiv  Detail & Related papers  (2021-02-01T20:58:45Z)
• Learning Equivariant Representations [10.745691354609738]
  Convolutional neural networks (CNNs) are successful examples of this principle. We propose equivariant models for different transformations defined by groups of symmetries. These models leverage symmetries in the data to reduce sample and model complexity and improve generalization performance.
  arXiv  Detail & Related papers  (2020-12-04T18:46:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.