Interpolated SelectionConv for Spherical Images and Surfaces

• URL: http://arxiv.org/abs/2210.10123v1
• Date: Tue, 18 Oct 2022 19:49:07 GMT
• Title: Interpolated SelectionConv for Spherical Images and Surfaces
• Authors: David Hart, Michael Whitney, Bryan Morse
• Abstract summary: We present a new and general framework for convolutional neural network operations on spherical images. Our approach represents the surface as a graph of connected points that doesn't rely on a particular sampling strategy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a new and general framework for convolutional neural network operations on spherical (or omnidirectional) images. Our approach represents the surface as a graph of connected points that doesn't rely on a particular sampling strategy. Additionally, by using an interpolated version of SelectionConv, we can operate on the sphere while using existing 2D CNNs and their weights. Since our method leverages existing graph implementations, it is also fast and can be fine-tuned efficiently. Our method is also general enough to be applied to any surface type, even those that are topologically non-simple. We demonstrate the effectiveness of our technique on the tasks of style transfer and segmentation for spheres as well as stylization for 3D meshes. We provide a thorough ablation study of the performance of various spherical sampling strategies.

Related papers

• GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs [49.55919802779889]
  We propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion. In this work, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning. Our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps.
  arXiv  Detail & Related papers  (2022-10-19T17:56:03Z)
• Neural Convolutional Surfaces [59.172308741945336]
  This work is concerned with a representation of shapes that disentangles fine, local and possibly repeating geometry, from global, coarse structures. We show that this approach achieves better neural shape compression than the state of the art, as well as enabling manipulation and transfer of shape details.
  arXiv  Detail & Related papers  (2022-04-05T15:40:11Z)
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression [59.58879331876508]
  We study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. Our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images.
  arXiv  Detail & Related papers  (2021-07-19T22:14:30Z)
• Concentric Spherical GNN for 3D Representation Learning [53.45704095146161]
  We propose a novel multi-resolution convolutional architecture for learning over concentric spherical feature maps. Our hierarchical architecture is based on alternatively learning to incorporate both intra-sphere and inter-sphere information. We demonstrate the effectiveness of our approach in improving state-of-the-art performance on 3D classification tasks with rotated data.
  arXiv  Detail & Related papers  (2021-03-18T19:05:04Z)
• Spherical Transformer: Adapting Spherical Signal to CNNs [53.18482213611481]
  Spherical Transformer can transform spherical signals into vectors that can be directly processed by standard CNNs. We evaluate our approach on the tasks of spherical MNIST recognition, 3D object classification and omnidirectional image semantic segmentation.
  arXiv  Detail & Related papers  (2021-01-11T12:33:16Z)
• DiffusionNet: Discretization Agnostic Learning on Surfaces [48.658589779470454]
  We introduce a new approach to deep learning on 3D surfaces, based on the insight that a simple diffusion layer is highly effective for spatial communication. The resulting networks automatically generalize across different samplings and resolutions of a surface. We focus primarily on triangle mesh surfaces, and demonstrate state-of-the-art results for a variety of tasks including surface classification, segmentation, and non-rigid correspondence.
  arXiv  Detail & Related papers  (2020-12-01T23:24:22Z)
• Primal-Dual Mesh Convolutional Neural Networks [62.165239866312334]
  We propose a primal-dual framework drawn from the graph-neural-network literature to triangle meshes. Our method takes features for both edges and faces of a 3D mesh as input and dynamically aggregates them. We provide theoretical insights of our approach using tools from the mesh-simplification literature.
  arXiv  Detail & Related papers  (2020-10-23T14:49:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.