Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction

• URL: http://arxiv.org/abs/2506.04243v1
• Date: Wed, 28 May 2025 22:30:35 GMT
• Title: Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction
• Authors: Warayut Dokduea, Weerachart Tangchirapat, Sompote Youwai,
• Abstract summary: This paper presents a novel Triple Attention Transformer Architecture for predicting time-dependent concrete creep.<n>By transforming concrete creep prediction into an autoregressive sequence modeling task similar to language processing, our architecture leverages the transformer's self-attention mechanisms.<n>The architecture achieves exceptional performance with mean absolute percentage error of 1.63% and R2 values of 0.999 across all datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a novel Triple Attention Transformer Architecture for predicting time-dependent concrete creep, addressing fundamental limitations in current approaches that treat time as merely an input parameter rather than modeling the sequential nature of deformation development. By transforming concrete creep prediction into an autoregressive sequence modeling task similar to language processing, our architecture leverages the transformer's self-attention mechanisms to capture long-range dependencies in historical creep patterns. The model implements a triple-stream attention framework incorporating temporal attention for sequential progression, feature attention for material property interactions, and batch attention for inter-sample relationships. Evaluated on experimental datasets with standardized daily measurements spanning 160 days, the architecture achieves exceptional performance with mean absolute percentage error of 1.63% and R2 values of 0.999 across all datasets, substantially outperforming traditional empirical models and existing machine learning approaches. Ablation studies confirm the critical role of attention mechanisms, with attention pooling contributing most significantly to model performance. SHAP analysis reveals Young's modulus as the primary predictive feature, followed by density and compressive strength, providing interpretability essential for engineering applications. A deployed web-based interface facilitates practical implementation, enabling real-time predictions using standard laboratory parameters. This work establishes the viability of applying transformer architectures to materials science problems, demonstrating the potential for data-driven approaches to revolutionize structural behavior prediction and engineering design practices.

Related papers

• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [55.914891182214475]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• Synthetic Time Series Forecasting with Transformer Architectures: Extensive Simulation Benchmarks [1.03590082373586]
  Time series forecasting plays a critical role in domains such as energy, finance, and healthcare.<n>Autoformer, Informer, and Patchtst-each evaluated through three architectural variants.<n>Koopman-enhanced Transformer framework, Deep Koopformer, integrates operator-theoretic latent state modeling.
  arXiv  Detail & Related papers  (2025-05-26T14:34:05Z)
• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay.<n>Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks.<n>Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• WAVE: Weighted Autoregressive Varying Gate for Time Series Forecasting [9.114664059026767]
  We propose a weighted Autoregressive Varying gatE attention mechanism equipped with both Autoregressive (AR) and Moving-average (MA) components.<n>It can adapt to various attention mechanisms, enhancing and decoupling their ability to capture long-range and local temporal patterns in time series data.
  arXiv  Detail & Related papers  (2024-10-04T05:45:50Z)
• Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting [46.63798583414426]
  Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis. Our study demonstrates, through both analytical and empirical evidence, that decomposition is key to containing excessive model inflation. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks.
  arXiv  Detail & Related papers  (2024-01-22T13:15:40Z)
• Enhanced LFTSformer: A Novel Long-Term Financial Time Series Prediction Model Using Advanced Feature Engineering and the DS Encoder Informer Architecture [0.8532753451809455]
  This study presents a groundbreaking model for forecasting long-term financial time series, termed the Enhanced LFTSformer. The model distinguishes itself through several significant innovations. Systematic experimentation on a range of benchmark stock market datasets demonstrates that the Enhanced LFTSformer outperforms traditional machine learning models.
  arXiv  Detail & Related papers  (2023-10-03T08:37:21Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting [68.86835407617778]
  Autoformer is a novel decomposition architecture with an Auto-Correlation mechanism. In long-term forecasting, Autoformer yields state-of-the-art accuracy, with a relative improvement on six benchmarks.
  arXiv  Detail & Related papers  (2021-06-24T13:43:43Z)
• Attention Mechanism for Multivariate Time Series Recurrent Model Interpretability Applied to the Ironmaking Industry [0.0]
  This paper focuses on the development of an interpretable multivariate time series forecasting deep learning architecture for the temperature of the hot metal produced by a blast furnace. A Long Short-Term Memory (LSTM) based architecture is proposed to accommodate the prediction with a local temporal interpretability for each input. Results are showing high potential for this architecture applied to blast furnace data and providing interpretability correctly reflecting the true complex variables relations dictated by the inherent blast furnace process.
  arXiv  Detail & Related papers  (2020-07-15T07:18:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.