Circular-Symmetric Correlation Layer based on FFT

• URL: http://arxiv.org/abs/2107.12480v1
• Date: Mon, 26 Jul 2021 21:06:20 GMT
• Title: Circular-Symmetric Correlation Layer based on FFT
• Authors: Bahar Azari and Deniz Erdogmus
• Abstract summary: We propose a Circular-symmetric Correlation Layer (CCL) based on the formalism of roto-translation equivariant correlation on the continuous group $S1 times mathbbR$. We showcase the performance analysis of a general network equipped with CCL on various recognition and classification tasks and datasets.
• Score: 11.634729459989996
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the vast success of standard planar convolutional neural networks, they are not the most efficient choice for analyzing signals that lie on an arbitrarily curved manifold, such as a cylinder. The problem arises when one performs a planar projection of these signals and inevitably causes them to be distorted or broken where there is valuable information. We propose a Circular-symmetric Correlation Layer (CCL) based on the formalism of roto-translation equivariant correlation on the continuous group $S^1 \times \mathbb{R}$, and implement it efficiently using the well-known Fast Fourier Transform (FFT) algorithm. We showcase the performance analysis of a general network equipped with CCL on various recognition and classification tasks and datasets. The PyTorch package implementation of CCL is provided online.

Related papers

• Inferring Dynamic Networks from Marginals with Iterative Proportional Fitting [57.487936697747024]
  A common network inference problem, arising from real-world data constraints, is how to infer a dynamic network from its time-aggregated adjacency matrix. We introduce a principled algorithm that guarantees IPF converges under minimal changes to the network structure.
  arXiv  Detail & Related papers  (2024-02-28T20:24:56Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• A Framework for Analyzing Cross-correlators using Price's Theorem and Piecewise-Linear Decomposition [5.094549132183797]
  We present a general mathematical framework that allows us to analyze cross-correlators constructed using a mixture of piece-wise linear functions. We show that some of the most promising cross-correlators are based on Huber's loss functions, margin-propagation (MP) functions, and the log-sum-exp (LSE) functions.
  arXiv  Detail & Related papers  (2023-04-18T19:03:27Z)
• Faster Adaptive Federated Learning [84.38913517122619]
  Federated learning has attracted increasing attention with the emergence of distributed data. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
  arXiv  Detail & Related papers  (2022-12-02T05:07:50Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Accurate Discharge Coefficient Prediction of Streamlined Weirs by Coupling Linear Regression and Deep Convolutional Gated Recurrent Unit [2.4475596711637433]
  The present study proposes data-driven modeling techniques, as an alternative to CFD simulation, to predict the discharge coefficient based on an experimental dataset. It is found that the proposed three layer hierarchical DL algorithm consists of a convolutional layer coupled with two subsequent GRU levels, which is also hybridized with the LR method, leads to lower error metrics.
  arXiv  Detail & Related papers  (2022-04-12T01:59:36Z)
• A Fast Parallel Tensor Decomposition with Optimal Stochastic Gradient Descent: an Application in Structural Damage Identification [1.536989504296526]
  We propose a novel algorithm, FP-CPD, to parallelize the CANDECOMP/PARAFAC (CP) decomposition of a tensor $mathcalX in mathbbR I_1 times dots times I_N $.
  arXiv  Detail & Related papers  (2021-11-04T05:17:07Z)
• Unfolding Projection-free SDP Relaxation of Binary Graph Classifier via GDPA Linearization [59.87663954467815]
  Algorithm unfolding creates an interpretable and parsimonious neural network architecture by implementing each iteration of a model-based algorithm as a neural layer. In this paper, leveraging a recent linear algebraic theorem called Gershgorin disc perfect alignment (GDPA), we unroll a projection-free algorithm for semi-definite programming relaxation (SDR) of a binary graph. Experimental results show that our unrolled network outperformed pure model-based graph classifiers, and achieved comparable performance to pure data-driven networks but using far fewer parameters.
  arXiv  Detail & Related papers  (2021-09-10T07:01:15Z)
• Orthogonalizing Convolutional Layers with the Cayley Transform [83.73855414030646]
  We propose and evaluate an alternative approach to parameterize convolutional layers that are constrained to be orthogonal. We show that our method indeed preserves orthogonality to a high degree even for large convolutions.
  arXiv  Detail & Related papers  (2021-04-14T23:54:55Z)
• Applying Convolutional Neural Networks to Data on Unstructured Meshes with Space-Filling Curves [0.4588028371034407]
  This paper presents the first classical Convolutional Neural Network (CNN) that can be applied directly to data from unstructured finite element meshes or control volume grids. CNNs have been hugely influential in the areas of image classification and image compression. Unstructured meshes are frequently used to solve partial differential equations.
  arXiv  Detail & Related papers  (2020-11-24T01:03:50Z)
• Posterior Re-calibration for Imbalanced Datasets [33.379680556475314]
  Neural Networks can perform poorly when the training label distribution is heavily imbalanced. We derive a post-training prior rebalancing technique that can be solved through a KL-divergence based optimization. Our results on six different datasets and five different architectures show state of art accuracy.
  arXiv  Detail & Related papers  (2020-10-22T15:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.