On generalized residue network for deep learning of unknown dynamical
systems
- URL: http://arxiv.org/abs/2002.02528v1
- Date: Thu, 23 Jan 2020 01:50:22 GMT
- Title: On generalized residue network for deep learning of unknown dynamical
systems
- Authors: Zhen Chen and Dongbin Xiu
- Abstract summary: We present a general numerical approach for learning unknown dynamical systems using deep neural networks (DNNs)
Our method is built upon recent studies that identified the residue network (ResNet) as an effective neural network structure.
- Score: 3.350695583277162
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a general numerical approach for learning unknown dynamical
systems using deep neural networks (DNNs). Our method is built upon recent
studies that identified the residue network (ResNet) as an effective neural
network structure. In this paper, we present a generalized ResNet framework and
broadly define residue as the discrepancy between observation data and
prediction made by another model, which can be an existing coarse model or
reduced-order model. In this case, the generalized ResNet serves as a model
correction to the existing model and recovers the unresolved dynamics. When an
existing coarse model is not available, we present numerical strategies for
fast creation of coarse models, to be used in conjunction with the generalized
ResNet. These coarse models are constructed using the same data set and thus do
not require additional resources. The generalized ResNet is capable of learning
the underlying unknown equations and producing predictions with accuracy higher
than the standard ResNet structure. This is demonstrated via several numerical
examples, including long-term prediction of a chaotic system.
Related papers
- A method for quantifying the generalization capabilities of generative models for solving Ising models [5.699467840225041]
We use a Hamming distance regularizer to quantify the generalization capabilities of various network architectures combined with VAN.
We conduct numerical experiments on several network architectures combined with VAN, including feed-forward neural networks, recurrent neural networks, and graph neural networks.
Our method is of great significance for assisting in the Neural Architecture Search field of searching for the optimal network architectures when solving large-scale Ising models.
arXiv Detail & Related papers (2024-05-06T12:58:48Z) - How neural networks learn to classify chaotic time series [77.34726150561087]
We study the inner workings of neural networks trained to classify regular-versus-chaotic time series.
We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
arXiv Detail & Related papers (2023-06-04T08:53:27Z) - Generalization and Estimation Error Bounds for Model-based Neural
Networks [78.88759757988761]
We show that the generalization abilities of model-based networks for sparse recovery outperform those of regular ReLU networks.
We derive practical design rules that allow to construct model-based networks with guaranteed high generalization.
arXiv Detail & Related papers (2023-04-19T16:39:44Z) - An Interpretive Constrained Linear Model for ResNet and MgNet [4.407784399315197]
We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN)
We establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models.
We present some modified ResNet models that compared with the original models have fewer parameters and yet can produce more accurate results.
arXiv Detail & Related papers (2021-12-14T14:52:44Z) - Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for
sparse recover [87.28082715343896]
We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications.
We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem.
The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
arXiv Detail & Related papers (2021-10-20T06:15:45Z) - Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
We show that pruning improves generalization for neural ODEs in generative modeling.
We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
arXiv Detail & Related papers (2021-06-24T01:40:17Z) - Anomaly Detection on Attributed Networks via Contrastive Self-Supervised
Learning [50.24174211654775]
We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks.
Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair.
A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
arXiv Detail & Related papers (2021-02-27T03:17:20Z) - Firearm Detection via Convolutional Neural Networks: Comparing a
Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents.
One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis.
We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
arXiv Detail & Related papers (2020-12-17T15:19:29Z) - Interpolation between Residual and Non-Residual Networks [24.690238357686134]
We present a novel ODE model by adding a damping term.
It can be shown that the proposed model can recover both a ResNet and a CNN by adjusting an coefficient.
Experiments on a number of image classification benchmarks show that the proposed model substantially improves the accuracy of ResNet and ResNeXt.
arXiv Detail & Related papers (2020-06-10T09:36:38Z) - Learning Queuing Networks by Recurrent Neural Networks [0.0]
We propose a machine-learning approach to derive performance models from data.
We exploit a deterministic approximation of their average dynamics in terms of a compact system of ordinary differential equations.
This allows for an interpretable structure of the neural network, which can be trained from system measurements to yield a white-box parameterized model.
arXiv Detail & Related papers (2020-02-25T10:56:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.