Learnable Discrete Wavelet Pooling (LDW-Pooling) For Convolutional Networks

• URL: http://arxiv.org/abs/2109.06638v2
• Date: Wed, 15 Sep 2021 06:22:57 GMT
• Title: Learnable Discrete Wavelet Pooling (LDW-Pooling) For Convolutional Networks
• Authors: Jun-Wei Hsieh, Ming-Ching Chang, Bor-Shiun Wang, Ping-Yang Chen, Lipeng Ke, Siwei Lyu
• Abstract summary: We introduce the Learning Discrete Wavelet Pooling (LDW-Pooling) that can be applied universally to replace standard pooling operations. LDW-Pooling is effective and efficient when compared with other state-of-the-art pooling techniques such as WaveletPooling and LiftPooling.
• Score: 33.45407848136399
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pooling is a simple but essential layer in modern deep CNN architectures for feature aggregation and extraction. Typical CNN design focuses on the conv layers and activation functions, while leaving the pooling layers with fewer options. We introduce the Learning Discrete Wavelet Pooling (LDW-Pooling) that can be applied universally to replace standard pooling operations to better extract features with improved accuracy and efficiency. Motivated from the wavelet theory, we adopt the low-pass (L) and high-pass (H) filters horizontally and vertically for pooling on a 2D feature map. Feature signals are decomposed into four (LL, LH, HL, HH) subbands to retain features better and avoid information dropping. The wavelet transform ensures features after pooling can be fully preserved and recovered. We next adopt an energy-based attention learning to fine-select crucial and representative features. LDW-Pooling is effective and efficient when compared with other state-of-the-art pooling techniques such as WaveletPooling and LiftPooling. Extensive experimental validation shows that LDW-Pooling can be applied to a wide range of standard CNN architectures and consistently outperform standard (max, mean, mixed, and stochastic) pooling operations.

Related papers

• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• MorphPool: Efficient Non-linear Pooling & Unpooling in CNNs [9.656707333320037]
  Pooling is essentially an operation from the field of Mathematical Morphology, with max pooling as a limited special case. In addition to pooling operations, encoder-decoder networks used for pixel-level predictions also require unpooling. Extensive experimentation on two tasks and three large-scale datasets shows that morphological pooling and unpooling lead to improved predictive performance at much reduced parameter counts.
  arXiv  Detail & Related papers  (2022-11-25T11:25:20Z)
• Hierarchical Spherical CNNs with Lifting-based Adaptive Wavelets for Pooling and Unpooling [101.72318949104627]
  We propose a novel framework of hierarchical convolutional neural networks (HS-CNNs) with a lifting structure to learn adaptive spherical wavelets for pooling and unpooling. LiftHS-CNN ensures a more efficient hierarchical feature learning for both image- and pixel-level tasks.
  arXiv  Detail & Related papers  (2022-05-31T07:23:42Z)
• Revisiting Pooling through the Lens of Optimal Transport [25.309212446782684]
  We develop a novel and solid algorithmic pooling framework through the lens of optimal transport. We make the parameters of the UOT problem learnable, and accordingly, propose a generalized pooling layer called textitUOT-Pooling for neural networks. We test our UOT-Pooling layers in two application scenarios, including multi-instance learning (MIL) and graph embedding.
  arXiv  Detail & Related papers  (2022-01-23T06:20:39Z)
• AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling [82.08631594071656]
  Pooling layers are essential building blocks of Convolutional Neural Networks (CNNs) We propose an adaptive and exponentially weighted pooling method named adaPool. We demonstrate how adaPool improves the preservation of detail through a range of tasks including image and video classification and object detection.
  arXiv  Detail & Related papers  (2021-11-01T08:50:37Z)
• Refining activation downsampling with SoftPool [74.1840492087968]
  Convolutional Neural Networks (CNNs) use pooling to decrease the size of activation maps. We propose SoftPool: a fast and efficient method for exponentially weighted activation downsampling. We show that SoftPool can retain more information in the reduced activation maps.
  arXiv  Detail & Related papers  (2021-01-02T12:09:49Z)
• Strip Pooling: Rethinking Spatial Pooling for Scene Parsing [161.7521770950933]
  We introduce strip pooling, which considers a long but narrow kernel, i.e., 1xN or Nx1. We compare the performance of the proposed strip pooling and conventional spatial pooling techniques. Both novel pooling-based designs are lightweight and can serve as an efficient plug-and-play module in existing scene parsing networks.
  arXiv  Detail & Related papers  (2020-03-30T10:40:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.