Capsule Network is Not More Robust than Convolutional Network

• URL: http://arxiv.org/abs/2103.15459v1
• Date: Mon, 29 Mar 2021 09:47:00 GMT
• Title: Capsule Network is Not More Robust than Convolutional Network
• Authors: Jindong Gu, Volker Tresp, Han Hu
• Abstract summary: We study the special designs in CapsNet that differ from that of a ConvNet commonly used for image classification. The study reveals that some designs, which are thought critical to CapsNet, actually can harm its robustness. We propose enhanced ConvNets simply by introducing the essential components behind the CapsNet's success.
• Score: 21.55939814377377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Capsule Network is widely believed to be more robust than Convolutional Networks. However, there are no comprehensive comparisons between these two networks, and it is also unknown which components in the CapsNet affect its robustness. In this paper, we first carefully examine the special designs in CapsNet that differ from that of a ConvNet commonly used for image classification. The examination reveals five major new/different components in CapsNet: a transformation process, a dynamic routing layer, a squashing function, a marginal loss other than cross-entropy loss, and an additional class-conditional reconstruction loss for regularization. Along with these major differences, we conduct comprehensive ablation studies on three kinds of robustness, including affine transformation, overlapping digits, and semantic representation. The study reveals that some designs, which are thought critical to CapsNet, actually can harm its robustness, i.e., the dynamic routing layer and the transformation process, while others are beneficial for the robustness. Based on these findings, we propose enhanced ConvNets simply by introducing the essential components behind the CapsNet's success. The proposed simple ConvNets can achieve better robustness than the CapsNet.

Related papers

• Designing Concise ConvNets with Columnar Stages [33.248031676529635]
  We introduce a refreshing ConvNet macro design called Columnar Stage Network (CoSNet) CoSNet has a systematically developed simple and concise structure, smaller depth, low parameter count, low FLOPs, and attention-less operations. Our evaluations show that CoSNet rivals many renowned ConvNets and Transformer designs under resource-constrained scenarios.
  arXiv  Detail & Related papers  (2024-10-05T09:03:42Z)
• RobCaps: Evaluating the Robustness of Capsule Networks against Affine Transformations and Adversarial Attacks [11.302789770501303]
  Capsule Networks (CapsNets) are able to hierarchically preserve the pose relationships between multiple objects for image classification tasks. In this paper, we evaluate different factors affecting the robustness of CapsNets, compared to traditional Conal Neural Networks (CNNs)
  arXiv  Detail & Related papers  (2023-04-08T09:58:35Z)
• MogaNet: Multi-order Gated Aggregation Network [64.16774341908365]
  We propose a new family of modern ConvNets, dubbed MogaNet, for discriminative visual representation learning. MogaNet encapsulates conceptually simple yet effective convolutions and gated aggregation into a compact module. MogaNet exhibits great scalability, impressive efficiency of parameters, and competitive performance compared to state-of-the-art ViTs and ConvNets on ImageNet.
  arXiv  Detail & Related papers  (2022-11-07T04:31:17Z)
• Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers [83.74380713308605]
  We develop a new type of transformation that is fully compatible with a variant of ReLUs -- Leaky ReLUs. We show in experiments that our method, which introduces negligible extra computational cost, validation accuracies with deep vanilla networks that are competitive with ResNets.
  arXiv  Detail & Related papers  (2022-03-15T17:49:08Z)
• Parallel Capsule Networks for Classification of White Blood Cells [1.5749416770494706]
  Capsule Networks (CapsNets) is a machine learning architecture proposed to overcome some of the shortcomings of convolutional neural networks (CNNs) We present a new architecture, parallel CapsNets, which exploits the concept of branching the network to isolate certain capsules.
  arXiv  Detail & Related papers  (2021-08-05T14:30:44Z)
• Rethinking and Improving the Robustness of Image Style Transfer [70.86976598145111]
  We show that the correlation between features extracted by a pre-trained VGG network has a remarkable ability to capture the visual style of an image. This quality is not robust and often degrades significantly when applied to features from more advanced and lightweight networks. We propose a solution based on a softmax transformation of the feature activations that enhances their entropy.
  arXiv  Detail & Related papers  (2021-04-08T03:24:45Z)
• Pushing the Limits of Capsule Networks [1.8231854497751137]
  Convolutional neural networks do not explicitly maintain a representation of the locations of the features relative to each other. A team at Google Brain recently made news with an attempt to fix this problem: Capsule Networks. We want to stress test CapsNet in various incremental ways to better understand their performance and expressiveness.
  arXiv  Detail & Related papers  (2021-03-15T00:30:34Z)
• Interpretable Graph Capsule Networks for Object Recognition [17.62514568986647]
  We propose interpretable Graph Capsule Networks (GraCapsNets), where we replace the routing part with a multi-head attention-based Graph Pooling approach. GraCapsNets achieve better classification performance with fewer parameters and better adversarial robustness, when compared to CapsNets.
  arXiv  Detail & Related papers  (2020-12-03T03:18:00Z)
• ResNet or DenseNet? Introducing Dense Shortcuts to ResNet [80.35001540483789]
  This paper presents a unified perspective of dense summation to analyze them. We propose dense weighted normalized shortcuts as a solution to the dilemma between ResNet and DenseNet. Our proposed DSNet achieves significantly better results than ResNet, and achieves comparable performance as DenseNet but requiring fewer resources.
  arXiv  Detail & Related papers  (2020-10-23T16:00:15Z)
• Structured Convolutions for Efficient Neural Network Design [65.36569572213027]
  We tackle model efficiency by exploiting redundancy in the textitimplicit structure of the building blocks of convolutional neural networks. We show how this decomposition can be applied to 2D and 3D kernels as well as the fully-connected layers.
  arXiv  Detail & Related papers  (2020-08-06T04:38:38Z)
• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions [76.05981545084738]
  We propose several ideas for enhancing a binary network to close its accuracy gap from real-valued networks without incurring any additional computational cost. We first construct a baseline network by modifying and binarizing a compact real-valued network with parameter-free shortcuts. We show that the proposed ReActNet outperforms all the state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2020-03-07T02:12:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.