IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning

• URL: http://arxiv.org/abs/2603.20682v1
• Date: Sat, 21 Mar 2026 06:50:19 GMT
• Title: IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning
• Abstract summary: Capsule networks (CapsNets) are superior at modeling hierarchical spatial relationships but suffer from two critical limitations.<n>We propose IBCapsNet, a novel capsule architecture grounded in the Information Bottleneck (IB) principle.<n>Our work bridges information-theoretic representation learning with capsule networks, offering a principled path toward robust, efficient, and interpretable deep models.
• Score: 3.6912754756190345
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Capsule networks (CapsNets) are superior at modeling hierarchical spatial relationships but suffer from two critical limitations: high computational cost due to iterative dynamic routing and poor robustness under input corruptions. To address these issues, we propose IBCapsNet, a novel capsule architecture grounded in the Information Bottleneck (IB) principle. Instead of iterative routing, IBCapsNet employs a one-pass variational aggregation mechanism, where primary capsules are first compressed into a global context representation and then processed by class-specific variational autoencoders (VAEs) to infer latent capsules regularized by the KL divergence. This design enables efficient inference while inherently filtering out noise. Experiments on MNIST, Fashion-MNIST, SVHN and CIFAR-10 show that IBCapsNet matches CapsNet in clean-data accuracy (achieving 99.41% on MNIST and 92.01% on SVHN), yet significantly outperforms it under four types of synthetic noise - demonstrating average improvements of +17.10% and +14.54% for clamped additive and multiplicative noise, respectively. Moreover, IBCapsNet achieves 2.54x faster training and 3.64x higher inference throughput compared to CapsNet, while reducing model parameters by 4.66%. Our work bridges information-theoretic representation learning with capsule networks, offering a principled path toward robust, efficient, and interpretable deep models. Code is available at https://github.com/cxiang26/IBCapsnet

Related papers

• Automatic Pruning via Structured Lasso with Class-wise Information [21.801590100174902]
  We use structured lasso with guidance from Information Bottleneck theory to leverage precise class-wise information for model pruning.<n>Our approaches demonstrate superior performance across three datasets and six model architectures in extensive experiments.
  arXiv  Detail & Related papers  (2025-02-13T10:03:29Z)
• DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentation [8.240211805240023]
  We revisit the design of atrous convolutions in modern convolutional neural networks (CNNs) We propose DSNet, a Dual-Branch CNN architecture, which incorporates atrous convolutions in the shallow layers of the model architecture. Our models achieve a new state-of-the-art trade-off between accuracy and speed on ADE20K, Cityscapes and BDD datasets.
  arXiv  Detail & Related papers  (2024-06-06T02:51:57Z)
• OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning [21.5857226735951]
  Redundancy is a persistent challenge in Capsule Networks (CapsNet) We propose an Orthogonal Capsule Network (OrthCaps) to reduce redundancy, improve routing performance and decrease parameter counts.
  arXiv  Detail & Related papers  (2024-03-20T07:25:24Z)
• CFDP: Common Frequency Domain Pruning [0.3021678014343889]
  We introduce a novel end-to-end pipeline for model pruning via the frequency domain. We have achieved state-of-the-art results on CIFAR-10 with GoogLeNet reaching an accuracy of 95.25%, that is, +0.2% from the original model. In addition to notable performances, models produced via CFDP exhibit robustness to a variety of configurations.
  arXiv  Detail & Related papers  (2023-06-07T04:49:26Z)
• Attention-based Feature Compression for CNN Inference Offloading in Edge Computing [93.67044879636093]
  This paper studies the computational offloading of CNN inference in device-edge co-inference systems. We propose a novel autoencoder-based CNN architecture (AECNN) for effective feature extraction at end-device. Experiments show that AECNN can compress the intermediate data by more than 256x with only about 4% accuracy loss.
  arXiv  Detail & Related papers  (2022-11-24T18:10:01Z)
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers [105.74546828182834]
  We show a hardware-efficient dynamic inference regime, named dynamic weight slicing, which adaptively slice a part of network parameters for inputs with diverse difficulty levels. We present dynamic slimmable network (DS-Net) and dynamic slice-able network (DS-Net++) by input-dependently adjusting filter numbers of CNNs and multiple dimensions in both CNNs and transformers.
  arXiv  Detail & Related papers  (2021-09-21T09:57:21Z)
• Compact representations of convolutional neural networks via weight pruning and quantization [63.417651529192014]
  We propose a novel storage format for convolutional neural networks (CNNs) based on source coding and leveraging both weight pruning and quantization. We achieve a reduction of space occupancy up to 0.6% on fully connected layers and 5.44% on the whole network, while performing at least as competitive as the baseline.
  arXiv  Detail & Related papers  (2021-08-28T20:39:54Z)
• BWCP: Probabilistic Learning-to-Prune Channels for ConvNets via Batch Whitening [63.081808698068365]
  This work presents a probabilistic channel pruning method to accelerate Convolutional Neural Networks (CNNs) Previous pruning methods often zero out unimportant channels in training in a deterministic manner, which reduces CNN's learning capacity and results in suboptimal performance. We develop a probability-based pruning algorithm, called batch whitening channel pruning (BWCP), which canally discard unimportant channels by modeling the probability of a channel being activated.
  arXiv  Detail & Related papers  (2021-05-13T17:00:05Z)
• Network Pruning via Resource Reallocation [75.85066435085595]
  We propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL) PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings.
  arXiv  Detail & Related papers  (2021-03-02T16:28:10Z)
• ACP: Automatic Channel Pruning via Clustering and Swarm Intelligence Optimization for CNN [6.662639002101124]
  convolutional neural network (CNN) gets deeper and wider in recent years. Existing magnitude-based pruning methods are efficient, but the performance of the compressed network is unpredictable. We propose a novel automatic channel pruning method (ACP) ACP is evaluated against several state-of-the-art CNNs on three different classification datasets.
  arXiv  Detail & Related papers  (2021-01-16T08:56:38Z)
• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions [76.05981545084738]
  We propose several ideas for enhancing a binary network to close its accuracy gap from real-valued networks without incurring any additional computational cost. We first construct a baseline network by modifying and binarizing a compact real-valued network with parameter-free shortcuts. We show that the proposed ReActNet outperforms all the state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2020-03-07T02:12:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.