Hybrid Real- and Complex-valued Neural Network Architecture

• URL: http://arxiv.org/abs/2504.03497v1
• Date: Fri, 04 Apr 2025 14:52:44 GMT
• Title: Hybrid Real- and Complex-valued Neural Network Architecture
• Authors: Alex Young, Luan Vinícius Fiorio, Bo Yang, Boris Karanov, Wim van Houtum, Ronald M. Aarts,
• Abstract summary: We propose a emphhybrid real- and complex-valued emphneural network (HNN) architecture, designed to combine the computational efficiency of real-valued processing with the ability to handle complex-valued data.<n>Experiments with the AudioMNIST dataset demonstrate that the HNN reduces cross-entropy loss and consumes less parameters compared to an RVNN for all considered cases.
• Score: 2.6739705603496327
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a \emph{hybrid} real- and complex-valued \emph{neural network} (HNN) architecture, designed to combine the computational efficiency of real-valued processing with the ability to effectively handle complex-valued data. We illustrate the limitations of using real-valued neural networks (RVNNs) for inherently complex-valued problems by showing how it learnt to perform complex-valued convolution, but with notable inefficiencies stemming from its real-valued constraints. To create the HNN, we propose to use building blocks containing both real- and complex-valued paths, where information between domains is exchanged through domain conversion functions. We also introduce novel complex-valued activation functions, with higher generalisation and parameterisation efficiency. HNN-specific architecture search techniques are described to navigate the larger solution space. Experiments with the AudioMNIST dataset demonstrate that the HNN reduces cross-entropy loss and consumes less parameters compared to an RVNN for all considered cases. Such results highlight the potential for the use of partially complex-valued processing in neural networks and applications for HNNs in many signal processing domains.

Related papers

• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval.<n>A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed.<n>The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Steinmetz Neural Networks for Complex-Valued Data [23.80312814400945]
  We introduce a new approach to processing complex-valued data using DNNs consisting of parallel real-valuedworks with coupled outputs. Our proposed class of architectures, referred to as Steinmetz Neural Networks, incorporates multi-view learning to construct more interpretable representations in the latent space. Our numerical experiments depict the improved performance and robustness to additive noise, afforded by our proposed networks on benchmark datasets and synthetic examples.
  arXiv  Detail & Related papers  (2024-09-16T08:26:06Z)
• A simple algorithm for output range analysis for deep neural networks [0.0]
  This paper presents a novel approach for the output range estimation problem in Deep Neural Networks (DNNs) by integrating a Simulated Annealing (SA) algorithm.<n>The method effectively addresses the challenges by the lack of geometric information and non-linearity inherent inResNets.
  arXiv  Detail & Related papers  (2024-07-02T22:47:40Z)
• An Efficient Approach to Regression Problems with Tensor Neural Networks [5.345144592056051]
  This paper introduces a tensor neural network (TNN) to address nonparametric regression problems. The TNN demonstrates superior performance compared to conventional Feed-Forward Networks (FFN) and Radial Basis Function Networks (RBN) A significant innovation in our approach is the integration of statistical regression and numerical integration within the TNN framework.
  arXiv  Detail & Related papers  (2024-06-14T03:38:40Z)
• On the Computational Complexities of Complex-valued Neural Networks [0.0]
  Complex-valued neural networks (CVNNs) are nonlinear filters used in the digital signal processing of complex-domain data. This paper presents both the quantitative and computational complexities of CVNNs.
  arXiv  Detail & Related papers  (2023-10-19T18:14:04Z)
• Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection [95.84616822805664]
  We introduce CNNs-assisted Transformer architecture and propose a novel RGB-D SOD network with Point-aware Interaction and CNN-induced Refinement.<n>In order to alleviate the block effect and detail destruction problems brought by the Transformer naturally, we design a CNN-induced refinement (CNNR) unit for content refinement and supplementation.
  arXiv  Detail & Related papers  (2023-08-17T11:57:49Z)
• Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale [54.15522908057831]
  We propose an adapted version of the computationally-Mixer for STTD forecast at scale. Our results surprisingly show that this simple-yeteffective solution can rival SOTA baselines when tested on several traffic benchmarks. Our findings contribute to the exploration of simple-yet-effective models for real-world STTD forecasting.
  arXiv  Detail & Related papers  (2023-07-04T05:19:19Z)
• Rethinking complex-valued deep neural networks for monaural speech enhancement [22.033822936410246]
  We show that complex-valued deep neural networks (DNNs) do not provide a performance gain over their real-valued counterparts for monaural speech enhancement. We also find that the use of complex-valued operations hinders the model capacity when the model size is small.
  arXiv  Detail & Related papers  (2023-01-11T05:59:50Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision [64.71260357476602]
  Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks. We propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection.
  arXiv  Detail & Related papers  (2021-12-06T23:45:58Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.