Fully trainable Gaussian derivative convolutional layer

• URL: http://arxiv.org/abs/2207.08424v1
• Date: Mon, 18 Jul 2022 08:16:04 GMT
• Title: Fully trainable Gaussian derivative convolutional layer
• Abstract summary: We propose a high-level layer based on anisotropic, oriented and shifted Gaussian derivative kernels. The results show that the proposed layer has competitive performance compared to previous works. It can be successfully included in common deep architectures such as VGG16 for image classification and U-net for image segmentation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Gaussian kernel and its derivatives have already been employed for Convolutional Neural Networks in several previous works. Most of these papers proposed to compute filters by linearly combining one or several bases of fixed or slightly trainable Gaussian kernels with or without their derivatives. In this article, we propose a high-level configurable layer based on anisotropic, oriented and shifted Gaussian derivative kernels which generalize notions encountered in previous related works while keeping their main advantage. The results show that the proposed layer has competitive performance compared to previous works and that it can be successfully included in common deep architectures such as VGG16 for image classification and U-net for image segmentation.

Related papers

• Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations [0.46040036610482665]
  This paper presents an in-depth analysis of the scale generalisation properties of the scale-covariant and scale-invariant Gaussian derivative networks. Gamma derivative networks are evaluated on new rescaled versions of the Fashion-MNIST and the CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2024-09-17T12:51:04Z)
• Conformal Transformation of Kernels: A Geometric Perspective on Text Classification [44.99833362998488]
  We investigate the effects of conformal transformations on kernel functions used in Support Vector Machines. Our experiments were conducted on the Reuters dataset on two types of binary classification tasks. The findings indicate that conformal transformations can significantly improve kernel performance, particularly for sub-optimal kernels.
  arXiv  Detail & Related papers  (2024-06-01T17:01:01Z)
• Neural Tangent Kernels Motivate Graph Neural Networks with Cross-Covariance Graphs [94.44374472696272]
  We investigate NTKs and alignment in the context of graph neural networks (GNNs) Our results establish the theoretical guarantees on the optimality of the alignment for a two-layer GNN. These guarantees are characterized by the graph shift operator being a function of the cross-covariance between the input and the output data.
  arXiv  Detail & Related papers  (2023-10-16T19:54:21Z)
• A Unified Algebraic Perspective on Lipschitz Neural Networks [88.14073994459586]
  This paper introduces a novel perspective unifying various types of 1-Lipschitz neural networks. We show that many existing techniques can be derived and generalized via finding analytical solutions of a common semidefinite programming (SDP) condition. Our approach, called SDP-based Lipschitz Layers (SLL), allows us to design non-trivial yet efficient generalization of convex potential layers.
  arXiv  Detail & Related papers  (2023-03-06T14:31:09Z)
• GMConv: Modulating Effective Receptive Fields for Convolutional Kernels [52.50351140755224]
  In convolutional neural networks, the convolutions are performed using a square kernel with a fixed N $times$ N receptive field (RF) Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv) in this work. Our GMConv can directly replace the standard convolutions in existing CNNs and can be easily trained end-to-end by standard back-propagation.
  arXiv  Detail & Related papers  (2023-02-09T10:17:17Z)
• Learning with convolution and pooling operations in kernel methods [8.528384027684192]
  Recent empirical work has shown that hierarchical convolutional kernels improve the performance of kernel methods in image classification tasks. We study the precise interplay between approximation and generalization in convolutional architectures. Our results quantify how choosing an architecture adapted to the target function leads to a large improvement in the sample complexity.
  arXiv  Detail & Related papers  (2021-11-16T09:00:44Z)
• On Approximation in Deep Convolutional Networks: a Kernel Perspective [12.284934135116515]
  We study the success of deep convolutional networks on tasks involving high-dimensional data such as images or audio. We study this theoretically and empirically through the lens of kernel methods, by considering multi-layer convolutional kernels. We find that while expressive kernels operating on input patches are important at the first layer, simpler kernels can suffice in higher layers for good performance.
  arXiv  Detail & Related papers  (2021-02-19T17:03:42Z)
• Adaptive Convolution Kernel for Artificial Neural Networks [0.0]
  This paper describes a method for training the size of convolutional kernels to provide varying size kernels in a single layer. Experiments compared the proposed adaptive layers to ordinary convolution layers in a simple two-layer network. A segmentation experiment in the Oxford-Pets dataset demonstrated that replacing a single ordinary convolution layer in a U-shaped network with a single 7$times$7 adaptive layer can improve its learning performance and ability to generalize.
  arXiv  Detail & Related papers  (2020-09-14T12:36:50Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)
• DO-Conv: Depthwise Over-parameterized Convolutional Layer [66.46704754669169]
  We propose to augment a convolutional layer with an additional depthwise convolution, where each input channel is convolved with a different 2D kernel. We show with extensive experiments that the mere replacement of conventional convolutional layers with DO-Conv layers boosts the performance of CNNs.
  arXiv  Detail & Related papers  (2020-06-22T06:57:10Z)
• Infinitely Wide Graph Convolutional Networks: Semi-supervised Learning via Gaussian Processes [144.6048446370369]
  Graph convolutional neural networks(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification. We propose a GP regression model via GCNs(GPGC) for graph-based semi-supervised learning. We conduct extensive experiments to evaluate GPGC and demonstrate that it outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-26T10:02:32Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.