Complex-valued Adaptive System Identification via Low-Rank Tensor Decomposition

• URL: http://arxiv.org/abs/2306.16428v1
• Date: Wed, 28 Jun 2023 07:01:08 GMT
• Title: Complex-valued Adaptive System Identification via Low-Rank Tensor Decomposition
• Authors: Oliver Ploder, Christina Auer, Oliver Lang, Thomas Paireder, Mario Huemer
• Abstract summary: In this work we derive two new architectures to allow the processing of complex-valued signals. We show that these extensions are able to surpass the trivial, complex-valued extension of the original architecture in terms of performance.
• Score: 3.268878947476012
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Machine learning (ML) and tensor-based methods have been of significant interest for the scientific community for the last few decades. In a previous work we presented a novel tensor-based system identification framework to ease the computational burden of tensor-only architectures while still being able to achieve exceptionally good performance. However, the derived approach only allows to process real-valued problems and is therefore not directly applicable on a wide range of signal processing and communications problems, which often deal with complex-valued systems. In this work we therefore derive two new architectures to allow the processing of complex-valued signals, and show that these extensions are able to surpass the trivial, complex-valued extension of the original architecture in terms of performance, while only requiring a slight overhead in computational resources to allow for complex-valued operations.

Related papers

• TorchSISSO: A PyTorch-Based Implementation of the Sure Independence Screening and Sparsifying Operator for Efficient and Interpretable Model Discovery [0.0]
  Symbolic regression (SR) is a powerful machine learning approach that searches for both the structure and parameters of algebraic models. In this work, we introduce TorchSISSO, a native Python implementation built in the PyTorch framework. We demonstrate that TorchSISSO matches or exceeds the performance of the original SISSO across a range of tasks.
  arXiv  Detail & Related papers  (2024-10-02T17:02:17Z)
• Mechanistic Design and Scaling of Hybrid Architectures [114.3129802943915]
  We identify and test new hybrid architectures constructed from a variety of computational primitives. We experimentally validate the resulting architectures via an extensive compute-optimal and a new state-optimal scaling law analysis. We find MAD synthetics to correlate with compute-optimal perplexity, enabling accurate evaluation of new architectures.
  arXiv  Detail & Related papers  (2024-03-26T16:33:12Z)
• Building Blocks for a Complex-Valued Transformer Architecture [5.177947445379688]
  We aim to make deep learning applicable to complex-valued signals without using projections into $mathbbR2$. We present multiple versions of a complex-valued Scaled Dot-Product Attention mechanism as well as a complex-valued layer normalization. We test on a classification and a sequence generation task on the MusicNet dataset and show improved robustness to overfitting while maintaining on-par performance when compared to the real-valued transformer architecture.
  arXiv  Detail & Related papers  (2023-06-16T13:11:15Z)
• Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems [5.1672267755831705]
  We present a kernel family that incorporates a partitioning that is learnable via gradient-based methods. We empirically demonstrate excellent performance on various active learning tasks.
  arXiv  Detail & Related papers  (2023-03-17T14:50:51Z)
• Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information [77.19830787312743]
  In real-world reinforcement learning applications the learner's observation space is ubiquitously high-dimensional with both relevant and irrelevant information about the task at hand. We introduce a new problem setting for reinforcement learning, the Exogenous Decision Process (ExoMDP), in which the state space admits an (unknown) factorization into a small controllable component and a large irrelevant component. We provide a new algorithm, ExoRL, which learns a near-optimal policy with sample complexity in the size of the endogenous component.
  arXiv  Detail & Related papers  (2022-06-09T05:19:32Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONets [0.0]
  We propose a bi-fidelity modeling approach for complex physical systems. We model the discrepancy between the true system's response and low-fidelity response in the presence of a small training dataset. We apply the approach to model systems that have parametric uncertainty and are partially unknown.
  arXiv  Detail & Related papers  (2022-04-03T05:30:57Z)
• Deep Equilibrium Assisted Block Sparse Coding of Inter-dependent Signals: Application to Hyperspectral Imaging [71.57324258813675]
  A dataset of inter-dependent signals is defined as a matrix whose columns demonstrate strong dependencies. A neural network is employed to act as structure prior and reveal the underlying signal interdependencies. Deep unrolling and Deep equilibrium based algorithms are developed, forming highly interpretable and concise deep-learning-based architectures.
  arXiv  Detail & Related papers  (2022-03-29T21:00:39Z)
• X-volution: On the unification of convolution and self-attention [52.80459687846842]
  We propose a multi-branch elementary module composed of both convolution and self-attention operation. The proposed X-volution achieves highly competitive visual understanding improvements.
  arXiv  Detail & Related papers  (2021-06-04T04:32:02Z)
• Local Function Complexity for Active Learning via Mixture of Gaussian Processes [5.382740428160009]
  Inhomogeneities in real-world data, due to changes in the observation noise level or variations in the structural complexity of the source function, pose a unique set of challenges for statistical inference. In this paper, we draw on recent theoretical results on the estimation of local function complexity (LFC) We derive and estimate the Gaussian process regression (GPR)-based analog of the LPS-based LFC and use it as a substitute in the above framework to make it robust and scalable.
  arXiv  Detail & Related papers  (2019-02-27T17:55:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.