Theory and Implementation of Complex-Valued Neural Networks

• URL: http://arxiv.org/abs/2302.08286v1
• Date: Thu, 16 Feb 2023 13:31:10 GMT
• Title: Theory and Implementation of Complex-Valued Neural Networks
• Authors: Jose Agustin Barrachina, Chengfang Ren, Gilles Vieillard, Christele Morisseau, Jean-Philippe Ovarlez
• Abstract summary: This work explains in detail the theory behind Complex-Valued Neural Network (CVNN) It includes Wirtinger calculus, complex backpropagation, and basic modules such as complex layers. We also perform simulations on real-valued data, casting to the complex domain by means of the Hilbert Transform, and verifying the potential interest of CVNN even for non-complex data.
• Score: 9.6556424340252
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work explains in detail the theory behind Complex-Valued Neural Network (CVNN), including Wirtinger calculus, complex backpropagation, and basic modules such as complex layers, complex activation functions, or complex weight initialization. We also show the impact of not adapting the weight initialization correctly to the complex domain. This work presents a strong focus on the implementation of such modules on Python using cvnn toolbox. We also perform simulations on real-valued data, casting to the complex domain by means of the Hilbert Transform, and verifying the potential interest of CVNN even for non-complex data.

Related papers

• Comprehensive Survey of Complex-Valued Neural Networks: Insights into Backpropagation and Activation Functions [0.0]
  Despite the prevailing use of real-number implementations in current ANN frameworks, there is a growing interest in developing ANNs that utilize complex numbers. This paper presents a survey of recent advancements in complex-valued neural networks (CVNNs) We delve into the extension of the backpropagation algorithm to the complex domain, which enables the training of neural networks with complex-valued inputs, weights, AFs, and outputs.
  arXiv  Detail & Related papers  (2024-07-27T13:47:16Z)
• Defining Neural Network Architecture through Polytope Structures of Dataset [53.512432492636236]
  This paper defines upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question. We develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks. It is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces.
  arXiv  Detail & Related papers  (2024-02-04T08:57:42Z)
• Complex-valued Neural Networks -- Theory and Analysis [0.0]
  This work addresses different structures and classification of CVNNs. The theory behind complex activation functions, implications related to complex differentiability and special activations for CVNN output layers are presented. The objective of this work is to understand the dynamics and most recent developments of CVNNs.
  arXiv  Detail & Related papers  (2023-12-11T03:24:26Z)
• On the Computational Complexities of Complex-valued Neural Networks [0.0]
  Complex-valued neural networks (CVNNs) are nonlinear filters used in the digital signal processing of complex-domain data. This paper presents both the quantitative and computational complexities of CVNNs.
  arXiv  Detail & Related papers  (2023-10-19T18:14:04Z)
• A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms [64.3064050603721]
  We generalize Runge-Kutta neural network to a recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields similar iterations to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta solvers for ordinary differential equations.
  arXiv  Detail & Related papers  (2022-11-22T16:30:33Z)
• On Rademacher Complexity-based Generalization Bounds for Deep Learning [18.601449856300984]
  We show that the Rademacher complexity-based approach can generate non-vacuous generalisation bounds on Convolutional Neural Networks (CNNs) Our results show that the Rademacher complexity does not depend on the network length for CNNs with some special types of activation functions such as ReLU, Leaky ReLU, Parametric Rectifier Linear Unit, Sigmoid, and Tanh.
  arXiv  Detail & Related papers  (2022-08-08T17:24:04Z)
• Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration [77.99182201815763]
  Leveraging prior knowledge on intraclass variance due to transformations is a powerful method to improve the sample complexity of deep neural networks. We propose a novel monomial selection algorithm based on pruning methods to allow an application to more complex problems. We demonstrate the improved sample complexity on the Rotated-MNIST, SVHN and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2022-02-08T16:16:11Z)
• Spectral Complexity-scaled Generalization Bound of Complex-valued Neural Networks [78.64167379726163]
  This paper is the first work that proves a generalization bound for the complex-valued neural network. We conduct experiments by training complex-valued convolutional neural networks on different datasets.
  arXiv  Detail & Related papers  (2021-12-07T03:25:25Z)
• On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces [208.67848059021915]
  We study the exploration-exploitation tradeoff at the core of reinforcement learning. In particular, we prove that the complexity of the function class $mathcalF$ characterizes the complexity of the function. Our regret bounds are independent of the number of episodes.
  arXiv  Detail & Related papers  (2020-11-09T18:32:22Z)
• Measuring Model Complexity of Neural Networks with Curve Activation Functions [100.98319505253797]
  We propose the linear approximation neural network (LANN) to approximate a given deep model with curve activation function. We experimentally explore the training process of neural networks and detect overfitting. We find that the $L1$ and $L2$ regularizations suppress the increase of model complexity.
  arXiv  Detail & Related papers  (2020-06-16T07:38:06Z)
• Analysis of Deep Complex-Valued Convolutional Neural Networks for MRI Reconstruction [9.55767753037496]
  We investigate end-to-end complex-valued convolutional neural networks for image reconstruction in lieu of two-channel real-valued networks. We find that complex-valued CNNs with complex-valued convolutions provide superior reconstructions compared to real-valued convolutions with the same number of trainable parameters.
  arXiv  Detail & Related papers  (2020-04-03T19:00:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.