From Discrete to Continuous Convolution Layers

• URL: http://arxiv.org/abs/2006.11120v1
• Date: Fri, 19 Jun 2020 13:16:06 GMT
• Title: From Discrete to Continuous Convolution Layers
• Authors: Assaf Shocher and Ben Feinstein and Niv Haim and Michal Irani
• Abstract summary: We propose a generalization of the common Conv-layer, from a discrete layer to a Continuous Convolution layer. CC layers naturally extend Conv-layers by representing the filter as a learned continuous function over sub-pixel coordinates. This allows learnable and principled resizing of feature maps, to any size, dynamically and consistently across scales.
• Score: 14.41405579899976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A basic operation in Convolutional Neural Networks (CNNs) is spatial resizing of feature maps. This is done either by strided convolution (donwscaling) or transposed convolution (upscaling). Such operations are limited to a fixed filter moving at predetermined integer steps (strides). Spatial sizes of consecutive layers are related by integer scale factors, predetermined at architectural design, and remain fixed throughout training and inference time. We propose a generalization of the common Conv-layer, from a discrete layer to a Continuous Convolution (CC) Layer. CC Layers naturally extend Conv-layers by representing the filter as a learned continuous function over sub-pixel coordinates. This allows learnable and principled resizing of feature maps, to any size, dynamically and consistently across scales. Once trained, the CC layer can be used to output any scale/size chosen at inference time. The scale can be non-integer and differ between the axes. CC gives rise to new freedoms for architectural design, such as dynamic layer shapes at inference time, or gradual architectures where the size changes by a small factor at each layer. This gives rise to many desired CNN properties, new architectural design capabilities, and useful applications. We further show that current Conv-layers suffer from inherent misalignments, which are ameliorated by CC layers.

Related papers

• Towards a General Purpose CNN for Long Range Dependencies in $\mathrm{N}$D [49.57261544331683]
  We propose a single CNN architecture equipped with continuous convolutional kernels for tasks on arbitrary resolution, dimensionality and length without structural changes. We show the generality of our approach by applying the same CCNN to a wide set of tasks on sequential (1$mathrmD$) and visual data (2$mathrmD$) Our CCNN performs competitively and often outperforms the current state-of-the-art across all tasks considered.
  arXiv  Detail & Related papers  (2022-06-07T15:48:02Z)
• Learning strides in convolutional neural networks [34.20666933112202]
  This work introduces DiffStride, the first downsampling layer with learnable strides. Experiments on audio and image classification show the generality and effectiveness of our solution.
  arXiv  Detail & Related papers  (2022-02-03T16:03:36Z)
• Augmenting Convolutional networks with attention-based aggregation [55.97184767391253]
  We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning. We plug this learned aggregation layer with a simplistic patch-based convolutional network parametrized by 2 parameters (width and depth) It yields surprisingly competitive trade-offs between accuracy and complexity, in particular in terms of memory consumption.
  arXiv  Detail & Related papers  (2021-12-27T14:05:41Z)
• Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition [2.4240083226965115]
  We compress layers by generalizing the Kronecker Product Decomposition to apply to multidimensionals, leading to the Generalized Kronecker Product Decomposition(GKPD) Our approach yields a plug-and-play module that can be used as a drop-in replacement for any convolutional layer.
  arXiv  Detail & Related papers  (2021-09-29T20:45:08Z)
• EigenGAN: Layer-Wise Eigen-Learning for GANs [84.33920839885619]
  EigenGAN is able to unsupervisedly mine interpretable and controllable dimensions from different generator layers. By traversing the coefficient of a specific eigen-dimension, the generator can produce samples with continuous changes corresponding to a specific semantic attribute.
  arXiv  Detail & Related papers  (2021-04-26T11:14:37Z)
• Diverse Branch Block: Building a Convolution as an Inception-like Unit [123.59890802196797]
  We propose a universal building block of Convolutional Neural Network (ConvNet) to improve the performance without any inference-time costs. The Diverse Branch Block (DBB) enhances the representational capacity of a single convolution by combining diverse branches of different scales and complexities. After training, a DBB can be equivalently converted into a single conv layer for deployment.
  arXiv  Detail & Related papers  (2021-03-24T18:12:00Z)
• PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer [76.44375136492827]
  Convolutional Neural Networks (CNNs) are often scale-sensitive. We bridge this regret by exploiting multi-scale features in a finer granularity. The proposed convolution operation, named Poly-Scale Convolution (PSConv), mixes up a spectrum of dilation rates.
  arXiv  Detail & Related papers  (2020-07-13T05:14:11Z)
• DO-Conv: Depthwise Over-parameterized Convolutional Layer [66.46704754669169]
  We propose to augment a convolutional layer with an additional depthwise convolution, where each input channel is convolved with a different 2D kernel. We show with extensive experiments that the mere replacement of conventional convolutional layers with DO-Conv layers boosts the performance of CNNs.
  arXiv  Detail & Related papers  (2020-06-22T06:57:10Z)
• Non-Euclidean Universal Approximation [4.18804572788063]
  Modifications to a neural network's input and output layers are often required to accommodate the specificities of most practical learning tasks. We present general conditions describing feature and readout maps that preserve an architecture's ability to approximate any continuous functions uniformly on compacts.
  arXiv  Detail & Related papers  (2020-06-03T15:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.