Explainable Dual-Attention Tabular Transformer for Soil Electrical Resistivity Prediction: A Decision Support Framework for High-Voltage Substation Construction

• URL: http://arxiv.org/abs/2504.02834v1
• Date: Mon, 17 Mar 2025 04:30:32 GMT
• Title: Explainable Dual-Attention Tabular Transformer for Soil Electrical Resistivity Prediction: A Decision Support Framework for High-Voltage Substation Construction
• Abstract summary: This research introduces a novel dual-attention transformer architecture for predicting soil electrical resistivity.<n>The proposed architecture achieves superior predictive performance (Mean Absolute Percentage Error: 0.63%) compared to recent state of the art models.<n>We developes a web-based application implementing this model to provide engineers with an accessible decision support framework.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This research introduces a novel dual-attention transformer architecture for predicting soil electrical resistivity, a critical parameter for high-voltage substation construction. Our model employs attention mechanisms operating across both features and data batches, enhanced by feature embedding layers that project inputs into higher-dimensional spaces. We implements Particle Swarm Optimization for hyperparameter tuning, systematically optimizing embedding dimensions, attention heads, and neural network architecture. The proposed architecture achieves superior predictive performance (Mean Absolute Percentage Error: 0.63%) compared to recent state of the art models for tabular data. Crucially, our model maintains explainability through SHapley Additive exPlanations value analysis, revealing that fine particle content and dry density are the most influential parameters affecting soil resistivity. We developes a web-based application implementing this model to provide engineers with an accessible decision support framework that bridges geotechnical and electrical engineering requirements for the Electricity Generating Authority of Thailand. This integrated approach satisfies both structural stability and electrical safety standards, improving construction efficiency and safety compliance in high-voltage infrastructure implementation.

Related papers

• A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods [3.3180230163290463]
  Transformer-based approaches consistently outperform established methods, reducing forecast error by 6.6-10.7 %.<n>We evaluate Transformer-based time-series foundation model Chronos-2, which demonstrates competitive zero-shot performance on two datasets but fails to accurately capture special events in the TSO data.
  arXiv  Detail & Related papers  (2026-07-17T07:32:52Z)
• Lights, Camera, Carbon: Architectural Scaling Laws for Video Generation Energy Consumption [4.930883100315264]
  We present a framework for estimating the energy consumption of text-to-video (T2V) and text-to-video-audio (T2VA) models.<n> Forward, it predicts energy from generation parameters and architectural principles; backward, it recovers architectural scaling behavior from observed inference times.<n>This approach offers a standardized, empirically and theoretically grounded framework for sustainability benchmarking across T2V models and architectures.
  arXiv  Detail & Related papers  (2026-07-05T23:58:29Z)
• Unlocking Feature Learning in Gated Delta Networks at Scale [52.705562336559446]
  Training and scaling Large Language Models demand enormous computational resources.<n>We derive the scaling rules for Gated Delta Network.<n> Experiments on language-model pre-training confirm that our configurations enable stable learning-rate transfer.
  arXiv  Detail & Related papers  (2026-06-02T08:45:24Z)
• CatalyticMLLM: A Graph-Text Multimodal Large Language Model for Catalytic Materials [61.546562658308005]
  We propose QE-Catalytic-V2, a unified graph--text multimodal large language model for catalytic materials.<n>It integrates property prediction and inverse design within the same model and shared representation space.<n>Under this unified framework, QE-Catalytic-V2 can not only perform reliable property prediction but also generate and screen physically feasible CIF candidates.
  arXiv  Detail & Related papers  (2026-05-17T04:31:46Z)
• Physics-informed AI Accelerated Retention Analysis of Ferroelectric Vertical NAND: From Day-Scale TCAD to Second-Scale Surrogate Model [1.8836299537031278]
  Ferroelectric field-effect transistors (FeFET)-based vertical NAND (Fe-VNAND) has emerged as a promising candidate to overcome z-scaling limitations with lower programming voltages.<n>We present a Physics-Informed Neural Operator (PINO)-based AI surrogate model designed for high-efficiency prediction of threshold voltage (Vth) shifts and retention behavior.
  arXiv  Detail & Related papers  (2026-03-06T21:01:31Z)
• StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models [98.72926158261937]
  We propose a training-free token pruning framework for Visual AutoRegressive models.<n>We employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information.<n>To maintain valid next-scale prediction under sparse tokens, we introduce a nearest neighbor feature propagation strategy.
  arXiv  Detail & Related papers  (2026-03-02T11:35:05Z)
• Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting [63.8116386935854]
  We demonstrate that state-of-the-art probabilistic skill requires neither intricate architectural constraints nor specialized trainings.<n>We introduce a scalable framework for learning multi-scale atmospheric dynamics by combining a directly downsampled latent space with a history-conditioned local projector.<n>We find that our framework design is robust to the choice of probabilistic estimators, seamlessly supporting interpolants, diffusion models, and CRPS-based ensemble training.
  arXiv  Detail & Related papers  (2026-01-26T03:52:16Z)
• Weight Spectra Induced Efficient Model Adaptation [54.8615621415845]
  Fine-tuning large-scale foundation models incurs prohibitive computational costs.<n>We show that fine-tuning predominantly amplifies the top singular values while leaving the remainder largely intact.<n>We propose a novel method that leverages learnable rescaling of top singular directions.
  arXiv  Detail & Related papers  (2025-05-29T05:03:29Z)
• Fine-tune Smarter, Not Harder: Parameter-Efficient Fine-Tuning for Geospatial Foundation Models [16.522696273752835]
  Earth observation is crucial for monitoring environmental changes, responding to disasters, and managing natural resources. Foundation models facilitate remote sensing image analysis to retrieve relevant geoinformation accurately and efficiently. As these models grow in size, fine-tuning becomes increasingly challenging due to associated computational resources and costs.
  arXiv  Detail & Related papers  (2025-04-24T09:37:02Z)
• Offline Model-Based Optimization: Comprehensive Review [61.91350077539443]
  offline optimization is a fundamental challenge in science and engineering, where the goal is to optimize black-box functions using only offline datasets.<n>Recent advances in model-based optimization have harnessed the generalization capabilities of deep neural networks to develop offline-specific surrogate and generative models.<n>Despite its growing impact in accelerating scientific discovery, the field lacks a comprehensive review.
  arXiv  Detail & Related papers  (2025-03-21T16:35:02Z)
• Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems [65.22300383287904]
  Industrial Cyber-Physical Systems (ICPSs) are an integral component of modern manufacturing and industries.<n>By digitizing data throughout product life cycles, Digital Twins (DTs) in ICPSs enable a shift from current industrial infrastructures to intelligent and adaptive infrastructures.<n>GenAI can drive the construction and update of DTs to improve predictive accuracy and prepare for diverse smart manufacturing.
  arXiv  Detail & Related papers  (2024-08-02T10:47:10Z)
• Global Transformer Architecture for Indoor Room Temperature Forecasting [49.32130498861987]
  This work presents a global Transformer architecture for indoor temperature forecasting in multi-room buildings. It aims at optimizing energy consumption and reducing greenhouse gas emissions associated with HVAC systems. Notably, this study is the first to apply a Transformer architecture for indoor temperature forecasting in multi-room buildings.
  arXiv  Detail & Related papers  (2023-10-31T14:09:32Z)
• Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture [68.13678918660872]
  We design a more capable parameter-sharing architecture based on matrix product operator (MPO) MPO decomposition can reorganize and factorize the information of a parameter matrix into two parts. Our architecture shares the central tensor across all layers for reducing the model size.
  arXiv  Detail & Related papers  (2023-03-27T02:34:09Z)
• Tailoring potentials by simulation-aided design of gate layouts for spin qubit applications [0.4276883312743397]
  Gate-s of spin qubit devices are commonly adapted from previous successful devices. We present a general approach for electrostatically modelling new spin qubit device layouts.
  arXiv  Detail & Related papers  (2023-03-23T15:36:32Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• Design Space Exploration and Explanation via Conditional Variational Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
  This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE) The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
  arXiv  Detail & Related papers  (2022-11-29T17:28:31Z)
• Machine learning based surrogate modeling with SVD enabled training for nonlinear civil structures subject to dynamic loading [0.0]
  This paper proposes a machine learning based surrogate model framework to predict earthquakes. The framework is validated by using it to successfully predict the peak response of one-story and three-story buildings.
  arXiv  Detail & Related papers  (2022-06-12T11:57:58Z)
• An Intelligent End-to-End Neural Architecture Search Framework for Electricity Forecasting Model Development [4.940941112226529]
  We propose an intelligent automated architecture search (IAAS) framework for the development of time-series electricity forecasting models. The proposed framework contains three primary components, i.e., network function-preserving transformation operation, reinforcement learning (RL)-based network transformation control, and network screening. We demonstrate that the proposed IAAS framework significantly outperforms the ten existing models or methods in terms of forecasting accuracy and stability.
  arXiv  Detail & Related papers  (2022-03-25T10:36:27Z)
• Short-Term Load Forecasting using Bi-directional Sequential Models and Feature Engineering for Small Datasets [6.619735628398446]
  This paper presents a deep learning architecture for short-term load forecasting based on bidirectional sequential models. In the proposed architecture, the raw input and hand-crafted features are trained at separate levels and then their respective outputs are combined to make the final prediction. The efficacy of the proposed methodology is evaluated on datasets from five countries with completely different patterns.
  arXiv  Detail & Related papers  (2020-11-28T14:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.