Distributed Learning and its Application for Time-Series Prediction
- URL: http://arxiv.org/abs/2106.03211v1
- Date: Sun, 6 Jun 2021 18:57:30 GMT
- Title: Distributed Learning and its Application for Time-Series Prediction
- Authors: Nhuong V. Nguyen and Sybille Legitime
- Abstract summary: Extreme events are occurrences whose magnitude and potential cause extensive damage on people, infrastructure, and the environment.
Motivated by the extreme nature of the current global health landscape, which is plagued by the coronavirus pandemic, we seek to better understand and model extreme events.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Extreme events are occurrences whose magnitude and potential cause extensive
damage on people, infrastructure, and the environment. Motivated by the extreme
nature of the current global health landscape, which is plagued by the
coronavirus pandemic, we seek to better understand and model extreme events.
Modeling extreme events is common in practice and plays an important role in
time-series prediction applications. Our goal is to (i) compare and investigate
the effect of some common extreme events modeling methods to explore which
method can be practical in reality and (ii) accelerate the deep learning
training process, which commonly uses deep recurrent neural network (RNN), by
implementing the asynchronous local Stochastic Gradient Descent (SGD) framework
among multiple compute nodes. In order to verify our distributed extreme events
modeling, we evaluate our proposed framework on a stock data set S\&P500, with
a standard recurrent neural network. Our intuition is to explore the (best)
extreme events modeling method which could work well under the distributed deep
learning setting. Moreover, by using asynchronous distributed learning, we aim
to significantly reduce the communication cost among the compute nodes and
central server, which is the main bottleneck of almost all distributed learning
frameworks.
We implement our proposed work and evaluate its performance on representative
data sets, such as S\&P500 stock in $5$-year period. The experimental results
validate the correctness of the design principle and show a significant
training duration reduction upto $8$x, compared to the baseline single compute
node. Our results also show that our proposed work can achieve the same level
of test accuracy, compared to the baseline setting.
Related papers
- Neural Networks with LSTM and GRU in Modeling Active Fires in the Amazon [0.0]
This study presents a comprehensive methodology for modeling and forecasting the historical time series of active fire spots detected by the AQUA_M-T satellite in the Amazon, Brazil.
The approach employs a mixed Recurrent Neural Network (RNN) model, combining Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures to predict the monthly accumulations of daily detected active fire spots.
arXiv Detail & Related papers (2024-09-04T13:11:59Z) - Towards Flexible Time-to-event Modeling: Optimizing Neural Networks via
Rank Regression [17.684526928033065]
We introduce the Deep AFT Rank-regression model for Time-to-event prediction (DART)
This model uses an objective function based on Gehan's rank statistic, which is efficient and reliable for representation learning.
The proposed method is a semiparametric approach to AFT modeling that does not impose any distributional assumptions on the survival time distribution.
arXiv Detail & Related papers (2023-07-16T13:58:28Z) - Towards Out-of-Distribution Sequential Event Prediction: A Causal
Treatment [72.50906475214457]
The goal of sequential event prediction is to estimate the next event based on a sequence of historical events.
In practice, the next-event prediction models are trained with sequential data collected at one time.
We propose a framework with hierarchical branching structures for learning context-specific representations.
arXiv Detail & Related papers (2022-10-24T07:54:13Z) - Less is More: Rethinking Few-Shot Learning and Recurrent Neural Nets [2.824895388993495]
We provide theoretical guarantees for reliable learning under the information-theoretic AEP.
We then focus on a highly efficient recurrent neural net (RNN) framework and propose a reduced-entropy algorithm for few-shot learning.
Our experimental results demonstrate significant potential for improving learning models' sample efficiency, generalization, and time complexity.
arXiv Detail & Related papers (2022-09-28T17:33:11Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Probabilistic AutoRegressive Neural Networks for Accurate Long-range
Forecasting [6.295157260756792]
We introduce the Probabilistic AutoRegressive Neural Networks (PARNN)
PARNN is capable of handling complex time series data exhibiting non-stationarity, nonlinearity, non-seasonality, long-range dependence, and chaotic patterns.
We evaluate the performance of PARNN against standard statistical, machine learning, and deep learning models, including Transformers, NBeats, and DeepAR.
arXiv Detail & Related papers (2022-04-01T17:57:36Z) - Self-Ensembling GAN for Cross-Domain Semantic Segmentation [107.27377745720243]
This paper proposes a self-ensembling generative adversarial network (SE-GAN) exploiting cross-domain data for semantic segmentation.
In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN.
Despite its simplicity, we find SE-GAN can significantly boost the performance of adversarial training and enhance the stability of the model.
arXiv Detail & Related papers (2021-12-15T09:50:25Z) - Model-assisted deep learning of rare extreme events from partial
observations [0.0]
To predict rare extreme events using deep neural networks, one encounters the so-called small data problem.
Here, we investigate a model-assisted framework where the training data is obtained from numerical simulations.
We find that long short-term memory networks are most robust to noise and to yield relatively accurate predictions.
arXiv Detail & Related papers (2021-11-04T23:24:22Z) - A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via
Adversarial Fine-tuning [90.44219200633286]
We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy.
Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
arXiv Detail & Related papers (2020-12-25T20:50:15Z) - Statistical model-based evaluation of neural networks [74.10854783437351]
We develop an experimental setup for the evaluation of neural networks (NNs)
The setup helps to benchmark a set of NNs vis-a-vis minimum-mean-square-error (MMSE) performance bounds.
This allows us to test the effects of training data size, data dimension, data geometry, noise, and mismatch between training and testing conditions.
arXiv Detail & Related papers (2020-11-18T00:33:24Z) - A Multi-Channel Neural Graphical Event Model with Negative Evidence [76.51278722190607]
Event datasets are sequences of events of various types occurring irregularly over the time-line.
We propose a non-parametric deep neural network approach in order to estimate the underlying intensity functions.
arXiv Detail & Related papers (2020-02-21T23:10:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.