Convolutional Neural Networks Rarely Learn Shape for Semantic Segmentation

• URL: http://arxiv.org/abs/2305.06568v3
• Date: Mon, 27 May 2024 16:11:00 GMT
• Title: Convolutional Neural Networks Rarely Learn Shape for Semantic Segmentation
• Authors: Yixin Zhang, Maciej A. Mazurowski,
• Abstract summary: Shape learning could be a desirable property of convolutional neural networks (CNNs) when target objects have specific shapes. We present a new behavioral metric to measure the extent to which a CNN utilizes shape information. We conclude that CNNs do not learn shape in typical settings but rather rely on other features available to identify the objects of interest.
• Score: 16.343080265661882
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Shape learning, or the ability to leverage shape information, could be a desirable property of convolutional neural networks (CNNs) when target objects have specific shapes. While some research on the topic is emerging, there is no systematic study to conclusively determine whether and under what circumstances CNNs learn shape. Here, we present such a study in the context of segmentation networks where shapes are particularly important. We define shape and propose a new behavioral metric to measure the extent to which a CNN utilizes shape information. We then execute a set of experiments with synthetic and real-world data to progressively uncover under which circumstances CNNs learn shape and what can be done to encourage such behavior. We conclude that (i) CNNs do not learn shape in typical settings but rather rely on other features available to identify the objects of interest, (ii) CNNs can learn shape, but only if the shape is the only feature available to identify the object, (iii) sufficiently large receptive field size relative to the size of target objects is necessary for shape learning; (iv) a limited set of augmentations can encourage shape learning; (v) learning shape is indeed useful in the presence of out-of-distribution data.

Related papers

• A novel feature-scrambling approach reveals the capacity of convolutional neural networks to learn spatial relations [0.0]
  Convolutional neural networks (CNNs) are one of the most successful computer vision systems to solve object recognition. Yet it remains poorly understood how CNNs actually make their decisions, what the nature of their internal representations is, and how their recognition strategies differ from humans.
  arXiv  Detail & Related papers  (2022-12-12T16:40:29Z)
• SurFit: Learning to Fit Surfaces Improves Few Shot Learning on Point Clouds [48.61222927399794]
  SurFit is a simple approach for label efficient learning of 3D shape segmentation networks. It is based on a self-supervised task of decomposing the surface of a 3D shape into geometric primitives.
  arXiv  Detail & Related papers  (2021-12-27T23:55:36Z)
• iSeg3D: An Interactive 3D Shape Segmentation Tool [48.784624011210475]
  We propose an effective annotation tool, named iSeg for 3D shape. Under our observation, most objects can be considered as the composition of finite primitive shapes. We train iSeg3D model on our built primitive-composed shape data to learn the geometric prior knowledge in a self-supervised manner.
  arXiv  Detail & Related papers  (2021-12-24T08:15:52Z)
• Reasoning-Modulated Representations [85.08205744191078]
  We study a common setting where our task is not purely opaque. Our approach paves the way for a new class of data-efficient representation learning.
  arXiv  Detail & Related papers  (2021-07-19T13:57:13Z)
• Adversarial Shape Learning for Building Extraction in VHR Remote Sensing Images [18.650642666164252]
  We propose an adversarial shape learning network (ASLNet) to model the building shape patterns. Experiments show that the proposed ASLNet improves both the pixel-based accuracy and the object-based measurements by a large margin.
  arXiv  Detail & Related papers  (2021-02-22T18:49:43Z)
• Shape or Texture: Understanding Discriminative Features in CNNs [28.513300496205044]
  Recent studies have shown that CNNs actually exhibit a texture bias' We show that a network learns the majority of overall shape information at the first few epochs of training. We also show that the encoding of shape does not imply the encoding of localized per-pixel semantic information.
  arXiv  Detail & Related papers  (2021-01-27T18:54:00Z)
• Teaching CNNs to mimic Human Visual Cognitive Process & regularise Texture-Shape bias [18.003188982585737]
  Recent experiments in computer vision demonstrate texture bias as the primary reason for supreme results in models employing Convolutional Neural Networks (CNNs) It is believed that the cost function forces the CNN to take a greedy approach and develop a proclivity for local information like texture to increase accuracy, thus failing to explore any global statistics. We propose CognitiveCNN, a new intuitive architecture, inspired from feature integration theory in psychology to utilise human interpretable feature like shape, texture, edges etc. to reconstruct, and classify the image.
  arXiv  Detail & Related papers  (2020-06-25T22:32:54Z)
• Shape-Oriented Convolution Neural Network for Point Cloud Analysis [59.405388577930616]
  Point cloud is a principal data structure adopted for 3D geometric information encoding. Shape-oriented message passing scheme dubbed ShapeConv is proposed to focus on the representation learning of the underlying shape formed by each local neighboring point.
  arXiv  Detail & Related papers  (2020-04-20T16:11:51Z)
• SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds [96.51884187479585]
  We propose a novel 3D shape signature to explore the shape information from point clouds. By incorporating operations of symmetry, convex hull and chebyshev fitting, the proposed shape sig-nature is not only compact and effective but also robust to the noise. Experiments show that the proposed method performs remarkably better than existing methods on two large-scale datasets.
  arXiv  Detail & Related papers  (2020-04-06T16:01:41Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.