Attention Mechanism for Multivariate Time Series Recurrent Model Interpretability Applied to the Ironmaking Industry

• URL: http://arxiv.org/abs/2007.12617v1
• Date: Wed, 15 Jul 2020 07:18:36 GMT
• Title: Attention Mechanism for Multivariate Time Series Recurrent Model Interpretability Applied to the Ironmaking Industry
• Authors: Cedric Schockaert, Reinhard Leperlier, Assaad Moawad
• Abstract summary: This paper focuses on the development of an interpretable multivariate time series forecasting deep learning architecture for the temperature of the hot metal produced by a blast furnace. A Long Short-Term Memory (LSTM) based architecture is proposed to accommodate the prediction with a local temporal interpretability for each input. Results are showing high potential for this architecture applied to blast furnace data and providing interpretability correctly reflecting the true complex variables relations dictated by the inherent blast furnace process.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data-driven model interpretability is a requirement to gain the acceptance of process engineers to rely on the prediction of a data-driven model to regulate industrial processes in the ironmaking industry. In the research presented in this paper, we focus on the development of an interpretable multivariate time series forecasting deep learning architecture for the temperature of the hot metal produced by a blast furnace. A Long Short-Term Memory (LSTM) based architecture enhanced with attention mechanism and guided backpropagation is proposed to accommodate the prediction with a local temporal interpretability for each input. Results are showing high potential for this architecture applied to blast furnace data and providing interpretability correctly reflecting the true complex variables relations dictated by the inherent blast furnace process, and with reduced prediction error compared to a recurrent-based deep learning architecture.

Related papers

• Mechanistic Design and Scaling of Hybrid Architectures [114.3129802943915]
  We identify and test new hybrid architectures constructed from a variety of computational primitives. We experimentally validate the resulting architectures via an extensive compute-optimal and a new state-optimal scaling law analysis. We find MAD synthetics to correlate with compute-optimal perplexity, enabling accurate evaluation of new architectures.
  arXiv  Detail & Related papers  (2024-03-26T16:33:12Z)
• Enhanced LFTSformer: A Novel Long-Term Financial Time Series Prediction Model Using Advanced Feature Engineering and the DS Encoder Informer Architecture [0.8532753451809455]
  This study presents a groundbreaking model for forecasting long-term financial time series, termed the Enhanced LFTSformer. The model distinguishes itself through several significant innovations. Systematic experimentation on a range of benchmark stock market datasets demonstrates that the Enhanced LFTSformer outperforms traditional machine learning models.
  arXiv  Detail & Related papers  (2023-10-03T08:37:21Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• Multi-Dimensional Self Attention based Approach for Remaining Useful Life Estimation [0.17205106391379021]
  Remaining Useful Life (RUL) estimation plays a critical role in Prognostics and Health Management (PHM) This paper carries out research into the remaining useful life prediction model for multi-sensor devices in the IIoT scenario. A data-driven approach for RUL estimation is proposed in this paper.
  arXiv  Detail & Related papers  (2022-12-12T08:50:27Z)
• Interpreting Machine Learning Models for Room Temperature Prediction in Non-domestic Buildings [0.0]
  This work presents an interpretable machine learning model aimed at predicting room temperature in non-domestic buildings. We demonstrate experimentally that the proposed model can accurately forecast room temperatures eight hours ahead in real-time.
  arXiv  Detail & Related papers  (2021-11-23T11:16:35Z)
• Tsformer: Time series Transformer for tourism demand forecasting [1.0883306593668278]
  We propose a time series Transformer (Tsformer) with the Transformer-Decoder architecture for tourism demand forecasting. The proposed Tsformer encodes long-term dependency with encoder, captures short-term dependency with decoder, and simplifies the attention interactions. Experiments conducted on the Jiuzhaigou valley and Siguniang mountain tourism demand datasets with other nine baseline methods indicate that the proposed Tsformer outperformed all baseline models.
  arXiv  Detail & Related papers  (2021-07-22T06:33:20Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)
• VAE-LIME: Deep Generative Model Based Approach for Local Data-Driven Model Interpretability Applied to the Ironmaking Industry [70.10343492784465]
  It is necessary to expose to the process engineer, not solely the model predictions, but also their interpretability. Model-agnostic local interpretability solutions based on LIME have recently emerged to improve the original method. We present in this paper a novel approach, VAE-LIME, for local interpretability of data-driven models forecasting the temperature of the hot metal produced by a blast furnace.
  arXiv  Detail & Related papers  (2020-07-15T07:07:07Z)
• Predictive modeling approaches in laser-based material processing [59.04160452043105]
  This study aims to automate and forecast the effect of laser processing on material structures. The focus is centred on the performance of representative statistical and machine learning algorithms. Results can set the basis for a systematic methodology towards reducing material design, testing and production cost.
  arXiv  Detail & Related papers  (2020-06-13T17:28:52Z)
• Transformer Hawkes Process [79.16290557505211]
  We propose a Transformer Hawkes Process (THP) model, which leverages the self-attention mechanism to capture long-term dependencies. THP outperforms existing models in terms of both likelihood and event prediction accuracy by a notable margin. We provide a concrete example, where THP achieves improved prediction performance for learning multiple point processes when incorporating their relational information.
  arXiv  Detail & Related papers  (2020-02-21T13:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.