LSTM Variants for Chaotic Dynamical Systems: An Empirical Study on the Lorenz Attractor

• URL: http://arxiv.org/abs/2606.22662v1
• Date: Sun, 21 Jun 2026 20:39:35 GMT
• Title: LSTM Variants for Chaotic Dynamical Systems: An Empirical Study on the Lorenz Attractor
• Abstract summary: We study seven recurrent and convolutional architectures for the AI-DEEDS 2026 Chaotic Systems Challenge.<n>We obtain leaderboard scores between 45.72 and 58.81, with the BiLSTM trained with Huber loss being the strongest configuration.<n>Two findings stand out: (i) adding additive attention to the unidirectional baseline degraded performance by over ten points, and (ii) prepending a CNN front-end to either an LSTM or a BiLSTM did not help and slightly hurt the score.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Forecasting chaotic dynamical systems such as the Lorenz attractor is notoriously difficult: small numerical errors are amplified exponentially over long autoregressive rollouts. We study seven recurrent and convolutional architectures for the AI-DEEDS 2026 Chaotic Systems Challenge: a vanilla LSTM, an LSTM with additive attention, a Bidirectional LSTM (BiLSTM), a BiLSTM trained with the Huber loss, a Temporal Convolutional Network (TCN), a CNN front-end followed by an LSTM, and a CNN front-end followed by a BiLSTM. All models share the same pre-processing, sequence length, and rollout procedure, isolating the contribution of each design choice. The challenge scores predictions on a 0-100 scale where higher is better. We obtain leaderboard scores between 45.72 and 58.81, with the BiLSTM trained with Huber loss being the strongest configuration. Two findings stand out: (i) adding additive attention to the unidirectional baseline degraded performance by over ten points, and (ii) prepending a CNN front-end to either an LSTM or a BiLSTM did not help and slightly hurt the score. Per-pair RMSE measurements confirm that the BiLSTM family generalizes better in the harder pairs (6-7), while the LSTM + Attention model collapses there (RMSE up to 8.94 on pair 6). We discuss why bidirectional context and a robust loss help in chaotic regimes while attention and CNN front-ends fail in this setting.

Related papers

• MIS-LSTM: Multichannel Image-Sequence LSTM for Sleep Quality and Stress Prediction [0.0]
  MIS-LSTM is a hybrid framework that joins CNN encoders with an LSTM sequence model for sleep quality and stress prediction.<n>UALRE is an uncertainty-aware ensemble that overrides lowconfidence majority votes with high-confidence individual predictions.
  arXiv  Detail & Related papers  (2025-09-14T12:19:04Z)
• Organ-Agents: Virtual Human Physiology Simulator via LLMs [66.40796430669158]
  Organ-Agents is a multi-agent framework that simulates human physiology via LLM-driven agents.<n>We curated data from 7,134 sepsis patients and 7,895 controls, generating high-resolution trajectories across 9 systems and 125 variables.<n>Organ-Agents achieved high simulation accuracy on 4,509 held-out patients, with per-system MSEs 0.16 and robustness across SOFA-based severity strata.
  arXiv  Detail & Related papers  (2025-08-20T01:58:45Z)
• VecLSTM: Trajectory Data Processing and Management for Activity Recognition through LSTM Vectorization and Database Integration [1.1701842638497677]
  VecLSTM is a novel framework that enhances the performance and efficiency of LSTM-based neural networks. VecLSTM incorporates vectorization layers, leveraging optimized mathematical operations to process input sequences more efficiently.
  arXiv  Detail & Related papers  (2024-09-28T06:22:44Z)
• Deep Learning Calabi-Yau four folds with hybrid and recurrent neural network architectures [0.0]
  We apply deep learning based on hybrid convolutional-recurrent and purely recurrent neural network architectures to a dataset of $h1,1, h2,1, h3,1, h2,2$.<n>Our best performing individual model is CNN-LSTM-400, a hybrid CNN-LSTM with the LSTM hidden size of 400.<n>Compared with the 72% training ratio, there is a significant improvement of accuracies, which reached 99.85%, 98.66%, 96.26%, 84.77% for the best individual model.
  arXiv  Detail & Related papers  (2024-05-27T17:55:05Z)
• Evaluation of QCNN-LSTM for Disability Forecasting in Multiple Sclerosis Using Sequential Multisequence MRI [0.0]
  Long ShortTerm Memory (LSTM) models were studied to provide sequential relationships for each timepoint in MRIs of patients with Multiple Sclerosis (MS) Our hypothesis is that quantum models will provide competitive performance.
  arXiv  Detail & Related papers  (2024-01-22T17:14:47Z)
• Short-Term Aggregated Residential Load Forecasting using BiLSTM and CNN-BiLSTM [0.3499870393443268]
  Short-term residential load forecasting has become an increasing area of focus. Recently deep neural networks have been leveraged to address this issue. This paper investigates the capabilities of a bidirectional long short-term memory (BiLSTM) and a convolutional neural network-based BiLSTM.
  arXiv  Detail & Related papers  (2023-02-10T03:11:02Z)
• Bayesian Neural Network Language Modeling for Speech Recognition [59.681758762712754]
  State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex. In this paper, an overarching full Bayesian learning framework is proposed to account for the underlying uncertainty in LSTM-RNN and Transformer LMs.
  arXiv  Detail & Related papers  (2022-08-28T17:50:19Z)
• End-to-end LSTM based estimation of volcano event epicenter localization [55.60116686945561]
  An end-to-end based LSTM scheme is proposed to address the problem of volcano event localization. LSTM was chosen due to its capability to capture the dynamics of time varying signals. Results show that the LSTM based architecture provided a success rate, i.e., an error smaller than 1.0Km, equal to 48.5%.
  arXiv  Detail & Related papers  (2021-10-27T17:11:33Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)
• High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model [46.34788932277904]
  We improve conventional hybrid LSTM acoustic models for high-accuracy and low-latency automatic speech recognition. To achieve high accuracy, we use a contextual layer trajectory LSTM (cltLSTM), which decouples the temporal modeling and target classification tasks. We further improve the training strategy with sequence-level teacher-student learning.
  arXiv  Detail & Related papers  (2020-03-17T00:52:11Z)
• Depth-Adaptive Graph Recurrent Network for Text Classification [71.20237659479703]
  Sentence-State LSTM (S-LSTM) is a powerful and high efficient graph recurrent network. We propose a depth-adaptive mechanism for the S-LSTM, which allows the model to learn how many computational steps to conduct for different words as required.
  arXiv  Detail & Related papers  (2020-02-29T03:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.