Neural Harmonium: An Interpretable Deep Structure for Nonlinear Dynamic
System Identification with Application to Audio Processing
- URL: http://arxiv.org/abs/2310.07032v1
- Date: Tue, 10 Oct 2023 21:32:15 GMT
- Title: Neural Harmonium: An Interpretable Deep Structure for Nonlinear Dynamic
System Identification with Application to Audio Processing
- Authors: Karim Helwani, Erfan Soltanmohammadi, Michael M. Goodwin
- Abstract summary: Interpretability helps us understand a model's ability to generalize and reveal its limitations.
We introduce a causal interpretable deep structure for modeling dynamic systems.
Our proposed model makes use of the harmonic analysis by modeling the system in a time-frequency domain.
- Score: 4.599180419117645
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Improving the interpretability of deep neural networks has recently gained
increased attention, especially when the power of deep learning is leveraged to
solve problems in physics. Interpretability helps us understand a model's
ability to generalize and reveal its limitations. In this paper, we introduce a
causal interpretable deep structure for modeling dynamic systems. Our proposed
model makes use of the harmonic analysis by modeling the system in a
time-frequency domain while maintaining high temporal and spectral resolution.
Moreover, the model is built in an order recursive manner which allows for
fast, robust, and exact second order optimization without the need for an
explicit Hessian calculation. To circumvent the resulting high dimensionality
of the building blocks of our system, a neural network is designed to identify
the frequency interdependencies. The proposed model is illustrated and
validated on nonlinear system identification problems as required for audio
signal processing tasks. Crowd-sourced experimentation contrasting the
performance of the proposed approach to other state-of-the-art solutions on an
acoustic echo cancellation scenario confirms the effectiveness of our method
for real-life applications.
Related papers
- Probabilistic Decomposed Linear Dynamical Systems for Robust Discovery of Latent Neural Dynamics [5.841659874892801]
Time-varying linear state-space models are powerful tools for obtaining mathematically interpretable representations of neural signals.
Existing methods for latent variable estimation are not robust to dynamical noise and system nonlinearity.
We propose a probabilistic approach to latent variable estimation in decomposed models that improves robustness against dynamical noise.
arXiv Detail & Related papers (2024-08-29T18:58:39Z) - Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising [54.110544509099526]
Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data.
We propose a hybrid convolution and attention network (HCANet) to enhance HSI denoising.
Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet.
arXiv Detail & Related papers (2024-03-15T07:18:43Z) - eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling [9.52474299688276]
We introduce a low-rank structured variational autoencoder framework for nonlinear state-space graphical models.
We show that our approach consistently demonstrates the ability to learn a more predictive generative model.
arXiv Detail & Related papers (2024-03-03T02:19:49Z) - Neural Modelling of Dynamic Systems with Time Delays Based on an
Adjusted NEAT Algorithm [0.0]
The proposed algorithm is based on a well-known NeuroEvolution of Augmenting Topologies (NEAT) algorithm.
The research involved an extended validation study based on data generated from a mathematical model of an exemplary system.
The obtaining simulation results demonstrate the high effectiveness of the devised neural (black-box) models of dynamic systems with time delays.
arXiv Detail & Related papers (2023-09-21T15:04:42Z) - On Robust Numerical Solver for ODE via Self-Attention Mechanism [82.95493796476767]
We explore training efficient and robust AI-enhanced numerical solvers with a small data size by mitigating intrinsic noise disturbances.
We first analyze the ability of the self-attention mechanism to regulate noise in supervised learning and then propose a simple-yet-effective numerical solver, Attr, which introduces an additive self-attention mechanism to the numerical solution of differential equations.
arXiv Detail & Related papers (2023-02-05T01:39:21Z) - Deep Equilibrium Assisted Block Sparse Coding of Inter-dependent
Signals: Application to Hyperspectral Imaging [71.57324258813675]
A dataset of inter-dependent signals is defined as a matrix whose columns demonstrate strong dependencies.
A neural network is employed to act as structure prior and reveal the underlying signal interdependencies.
Deep unrolling and Deep equilibrium based algorithms are developed, forming highly interpretable and concise deep-learning-based architectures.
arXiv Detail & Related papers (2022-03-29T21:00:39Z) - Supervised DKRC with Images for Offline System Identification [77.34726150561087]
Modern dynamical systems are becoming increasingly non-linear and complex.
There is a need for a framework to model these systems in a compact and comprehensive representation for prediction and control.
Our approach learns these basis functions using a supervised learning approach.
arXiv Detail & Related papers (2021-09-06T04:39:06Z) - Quadratic mutual information regularization in real-time deep CNN models [51.66271681532262]
Regularization method motivated by the Quadratic Mutual Information is proposed.
Experiments on various binary classification problems are performed, indicating the effectiveness of the proposed models.
arXiv Detail & Related papers (2021-08-26T13:14:24Z) - Reconstructing a dynamical system and forecasting time series by
self-consistent deep learning [4.947248396489835]
We introduce a self-consistent deep-learning framework for a noisy deterministic time series.
It provides unsupervised filtering, state-space reconstruction, identification of the underlying differential equations and forecasting.
arXiv Detail & Related papers (2021-08-04T06:10:58Z) - Neural Dynamic Mode Decomposition for End-to-End Modeling of Nonlinear
Dynamics [49.41640137945938]
We propose a neural dynamic mode decomposition for estimating a lift function based on neural networks.
With our proposed method, the forecast error is backpropagated through the neural networks and the spectral decomposition.
Our experiments demonstrate the effectiveness of our proposed method in terms of eigenvalue estimation and forecast performance.
arXiv Detail & Related papers (2020-12-11T08:34:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.