Geo-SIC: Learning Deformable Geometric Shapes in Deep Image Classifiers

• URL: http://arxiv.org/abs/2210.13704v1
• Date: Tue, 25 Oct 2022 01:55:17 GMT
• Title: Geo-SIC: Learning Deformable Geometric Shapes in Deep Image Classifiers
• Authors: Jian Wang, Miaomiao Zhang
• Abstract summary: This paper presents Geo-SIC, the first deep learning model to learn deformable shapes in a deformation space for an improved performance of image classification. We introduce a newly designed framework that (i) simultaneously derives features from both image and latent shape spaces with large intra-class variations. We develop a boosted classification network, equipped with an unsupervised learning of geometric shape representations.
• Score: 8.781861951759948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deformable shapes provide important and complex geometric features of objects presented in images. However, such information is oftentimes missing or underutilized as implicit knowledge in many image analysis tasks. This paper presents Geo-SIC, the first deep learning model to learn deformable shapes in a deformation space for an improved performance of image classification. We introduce a newly designed framework that (i) simultaneously derives features from both image and latent shape spaces with large intra-class variations; and (ii) gains increased model interpretability by allowing direct access to the underlying geometric features of image data. In particular, we develop a boosted classification network, equipped with an unsupervised learning of geometric shape representations characterized by diffeomorphic transformations within each class. In contrast to previous approaches using pre-extracted shapes, our model provides a more fundamental approach by naturally learning the most relevant shape features jointly with an image classifier. We demonstrate the effectiveness of our method on both simulated 2D images and real 3D brain magnetic resonance (MR) images. Experimental results show that our model substantially improves the image classification accuracy with an additional benefit of increased model interpretability. Our code is publicly available at https://github.com/jw4hv/Geo-SIC

Related papers

• Invariant Shape Representation Learning For Image Classification [41.610264291150706]
  In this paper, we introduce a novel framework that for the first time develops invariant shape representation learning (ISRL) Our model ISRL is designed to jointly capture invariant features in latent shape spaces parameterized by deformable transformations. By embedding the features that are invariant with regard to target variables in different environments, our model consistently offers more accurate predictions.
  arXiv  Detail & Related papers  (2024-11-19T03:39:43Z)
• Few-shot Shape Recognition by Learning Deep Shape-aware Features [39.66613852292122]
  We propose a fewshot shape descriptor (FSSD) to recognize object shapes given only one or a few samples. We employ an embedding module for FSSD to extract transformation-invariant shape features. Thirdly, we propose a decoding module to include the supervision of shape masks and edges and align the original and reconstructed shape features.
  arXiv  Detail & Related papers  (2023-12-03T08:12:23Z)
• Explorable Mesh Deformation Subspaces from Unstructured Generative Models [53.23510438769862]
  Deep generative models of 3D shapes often feature continuous latent spaces that can be used to explore potential variations. We present a method to explore variations among a given set of landmark shapes by constructing a mapping from an easily-navigable 2D exploration space to a subspace of a pre-trained generative model.
  arXiv  Detail & Related papers  (2023-10-11T18:53:57Z)
• SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction [2.2954246824369218]
  3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. We propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues.
  arXiv  Detail & Related papers  (2023-09-06T19:30:22Z)
• Improving Shape Awareness and Interpretability in Deep Networks Using Geometric Moments [0.0]
  Deep networks for image classification often rely more on texture information than object shape. This paper presents a deep-learning model inspired by geometric moments. We demonstrate the effectiveness of our method on standard image classification datasets.
  arXiv  Detail & Related papers  (2022-05-24T02:08:05Z)
• Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images [82.32776379815712]
  We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. We adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable function for test-time optimization.
  arXiv  Detail & Related papers  (2022-04-21T03:42:31Z)
• Self-Supervised Image Representation Learning with Geometric Set Consistency [50.12720780102395]
  We propose a method for self-supervised image representation learning under the guidance of 3D geometric consistency. Specifically, we introduce 3D geometric consistency into a contrastive learning framework to enforce the feature consistency within image views.
  arXiv  Detail & Related papers  (2022-03-29T08:57:33Z)
• Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian [58.704089101826774]
  We present a 3D-aware image deformation method with minimal restrictions on shape category and deformation type. We take a supervised learning-based approach to predict the shape Laplacian of the underlying volume of a 3D reconstruction represented as a point cloud. In the experiments, we present our results of deforming 2D character and clothed human images.
  arXiv  Detail & Related papers  (2022-03-29T04:57:18Z)
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression [59.58879331876508]
  We study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. Our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images.
  arXiv  Detail & Related papers  (2021-07-19T22:14:30Z)
• Joint Deep Multi-Graph Matching and 3D Geometry Learning from Inhomogeneous 2D Image Collections [57.60094385551773]
  We propose a trainable framework for learning a deformable 3D geometry model from inhomogeneous image collections. We in addition obtain the underlying 3D geometry of the objects depicted in the 2D images.
  arXiv  Detail & Related papers  (2021-03-31T17:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.