Extreme-Long-short Term Memory for Time-series Prediction
- URL: http://arxiv.org/abs/2210.08244v1
- Date: Sat, 15 Oct 2022 09:45:48 GMT
- Title: Extreme-Long-short Term Memory for Time-series Prediction
- Authors: Sida Xing, Feihu Han, Suiyang Khoo
- Abstract summary: Long Short-Term Memory (LSTM) is a new type of Recurrent Neural Networks (RNN)
In this paper, we propose an advanced LSTM algorithm, the Extreme Long Short-Term Memory (E-LSTM)
The new E-LSTM requires only 2 epochs to obtain the results of the 7th epoch traditional LSTM.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The emergence of Long Short-Term Memory (LSTM) solves the problems of
vanishing gradient and exploding gradient in traditional Recurrent Neural
Networks (RNN). LSTM, as a new type of RNN, has been widely used in various
fields, such as text prediction, Wind Speed Forecast, depression prediction by
EEG signals, etc. The results show that improving the efficiency of LSTM can
help to improve the efficiency in other application areas.
In this paper, we proposed an advanced LSTM algorithm, the Extreme Long
Short-Term Memory (E-LSTM), which adds the inverse matrix part of Extreme
Learning Machine (ELM) as a new "gate" into the structure of LSTM. This "gate"
preprocess a portion of the data and involves the processed data in the cell
update of the LSTM to obtain more accurate data with fewer training rounds,
thus reducing the overall training time.
In this research, the E-LSTM model is used for the text prediction task.
Experimental results showed that the E-LSTM sometimes takes longer to perform a
single training round, but when tested on a small data set, the new E-LSTM
requires only 2 epochs to obtain the results of the 7th epoch traditional LSTM.
Therefore, the E-LSTM retains the high accuracy of the traditional LSTM, whilst
also improving the training speed and the overall efficiency of the LSTM.
Related papers
- Kernel Corrector LSTM [1.034961673489652]
We propose a new RW-ML algorithm, Kernel Corrector LSTM (KcLSTM), that replaces the meta-learner of cLSTM with a simpler method: Kernel Smoothing.
We empirically evaluate the forecasting accuracy and the training time of the new algorithm and compare it with cLSTM and LSTM.
arXiv Detail & Related papers (2024-04-28T18:44:10Z) - Bayesian Neural Network Language Modeling for Speech Recognition [59.681758762712754]
State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex.
In this paper, an overarching full Bayesian learning framework is proposed to account for the underlying uncertainty in LSTM-RNN and Transformer LMs.
arXiv Detail & Related papers (2022-08-28T17:50:19Z) - Working Memory Connections for LSTM [51.742526187978726]
We show that Working Memory Connections constantly improve the performance of LSTMs on a variety of tasks.
Numerical results suggest that the cell state contains useful information that is worth including in the gate structure.
arXiv Detail & Related papers (2021-08-31T18:01:30Z) - Learning representations with end-to-end models for improved remaining
useful life prognostics [64.80885001058572]
The remaining Useful Life (RUL) of equipment is defined as the duration between the current time and its failure.
We propose an end-to-end deep learning model based on multi-layer perceptron and long short-term memory layers (LSTM) to predict the RUL.
We will discuss how the proposed end-to-end model is able to achieve such good results and compare it to other deep learning and state-of-the-art methods.
arXiv Detail & Related papers (2021-04-11T16:45:18Z) - Prediction of financial time series using LSTM and data denoising
methods [0.29923891863939933]
This paper proposes an ensemble method based on data denoising methods, including the wavelet transform (WT) and singular spectrum analysis (SSA)
As WT and SSA can extract useful information from the original sequence and avoid overfitting, the hybrid model can better grasp the sequence pattern of the closing price of the DJIA.
arXiv Detail & Related papers (2021-03-05T07:32:36Z) - Automatic Remaining Useful Life Estimation Framework with Embedded
Convolutional LSTM as the Backbone [5.927250637620123]
We propose a new LSTM variant called embedded convolutional LSTM (E NeuralTM)
In ETM a group of different 1D convolutions is embedded into the LSTM structure. Through this, the temporal information is preserved between and within windows.
We show the superiority of our proposed ETM approach over the state-of-the-art approaches on several widely used benchmark data sets for RUL Estimation.
arXiv Detail & Related papers (2020-08-10T08:34:20Z) - Future Vector Enhanced LSTM Language Model for LVCSR [67.03726018635174]
This paper proposes a novel enhanced long short-term memory (LSTM) LM using the future vector.
Experiments show that, the proposed new LSTM LM gets a better result on BLEU scores for long term sequence prediction.
Rescoring using both the new and conventional LSTM LMs can achieve a very large improvement on the word error rate.
arXiv Detail & Related papers (2020-07-31T08:38:56Z) - Object Tracking through Residual and Dense LSTMs [67.98948222599849]
Deep learning-based trackers based on LSTMs (Long Short-Term Memory) recurrent neural networks have emerged as a powerful alternative.
DenseLSTMs outperform Residual and regular LSTM, and offer a higher resilience to nuisances.
Our case study supports the adoption of residual-based RNNs for enhancing the robustness of other trackers.
arXiv Detail & Related papers (2020-06-22T08:20:17Z) - Sentiment Analysis Using Simplified Long Short-term Memory Recurrent
Neural Networks [1.5146765382501612]
We perform sentiment analysis on a GOP Debate Twitter dataset.
To speed up training and reduce the computational cost and time, six different parameter reduced slim versions of the LSTM model are proposed.
arXiv Detail & Related papers (2020-05-08T12:50:10Z) - Multi-level Binarized LSTM in EEG Classification for Wearable Devices [0.31498833540989407]
Long Short-Term Memory (LSTM) is widely used in various sequential applications.
Binary LSTMs are introduced to cope with this problem, however, they lead to significant accuracy loss in some application such as EEG classification.
We propose an efficient multi-level binarized LSTM which has significantly reduced computations whereas ensuring an accuracy pretty close to full precision LSTM.
arXiv Detail & Related papers (2020-04-19T17:48:55Z) - High-Accuracy and Low-Latency Speech Recognition with Two-Head
Contextual Layer Trajectory LSTM Model [46.34788932277904]
We improve conventional hybrid LSTM acoustic models for high-accuracy and low-latency automatic speech recognition.
To achieve high accuracy, we use a contextual layer trajectory LSTM (cltLSTM), which decouples the temporal modeling and target classification tasks.
We further improve the training strategy with sequence-level teacher-student learning.
arXiv Detail & Related papers (2020-03-17T00:52:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.