Transformer-Based Indirect Structural Health Monitoring of Rail Infrastructure with Attention-Driven Detection and Localization of Transient Defects

• URL: http://arxiv.org/abs/2510.07606v1
• Date: Wed, 08 Oct 2025 23:01:53 GMT
• Title: Transformer-Based Indirect Structural Health Monitoring of Rail Infrastructure with Attention-Driven Detection and Localization of Transient Defects
• Authors: Sizhe Ma, Katherine A. Flanigan, Mario Bergés, James D. Brooks,
• Abstract summary: We introduce an incremental synthetic data benchmark designed to evaluate model robustness against progressively complex challenges.<n>We evaluate several established unsupervised models alongside our proposed Attention-Focused Transformer.<n>Our proposed model achieves accuracy comparable to the state-of-the-art solution while demonstrating better inference speed.
• Score: 1.1782896991259
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Indirect structural health monitoring (iSHM) for broken rail detection using onboard sensors presents a cost-effective paradigm for railway track assessment, yet reliably detecting small, transient anomalies (2-10 cm) remains a significant challenge due to complex vehicle dynamics, signal noise, and the scarcity of labeled data limiting supervised approaches. This study addresses these issues through unsupervised deep learning. We introduce an incremental synthetic data benchmark designed to systematically evaluate model robustness against progressively complex challenges like speed variations, multi-channel inputs, and realistic noise patterns encountered in iSHM. Using this benchmark, we evaluate several established unsupervised models alongside our proposed Attention-Focused Transformer. Our model employs a self-attention mechanism, trained via reconstruction but innovatively deriving anomaly scores primarily from deviations in learned attention weights, aiming for both effectiveness and computational efficiency. Benchmarking results reveal that while transformer-based models generally outperform others, all tested models exhibit significant vulnerability to high-frequency localized noise, identifying this as a critical bottleneck for practical deployment. Notably, our proposed model achieves accuracy comparable to the state-of-the-art solution while demonstrating better inference speed. This highlights the crucial need for enhanced noise robustness in future iSHM models and positions our more efficient attention-based approach as a promising foundation for developing practical onboard anomaly detection systems.

Related papers

• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• Flexible Gravitational-Wave Parameter Estimation with Transformers [73.44614054040267]
  We introduce a flexible transformer-based architecture paired with a training strategy that enables adaptation to diverse analysis settings at inference time.<n>We demonstrate that a single flexible model -- called Dingo-T1 -- can analyze 48 gravitational-wave events from the third LIGO-Virgo-KAGRA Observing Run.
  arXiv  Detail & Related papers  (2025-12-02T17:49:08Z)
• DRTA: Dynamic Reward Scaling for Reinforcement Learning in Time Series Anomaly Detection [7.185726339205792]
  Anomaly detection in time series data is important for applications in finance, healthcare, sensor networks, and industrial monitoring.<n>We propose a reinforcement learning-based framework that integrates dynamic reward shaping, Variational Autoencoder (VAE), and active learning, called DRTA.<n>Our method uses an adaptive reward mechanism that balances exploration and exploitation by dynamically scaling the effect of VAE-based reconstruction error and classification rewards.
  arXiv  Detail & Related papers  (2025-08-25T20:39:49Z)
• Noise Robust One-Class Intrusion Detection on Dynamic Graphs [46.453758431767724]
  This study introduces a probabilistic version of the Temporal Graph Network Support Vector Data Description (TGN-SVDD) model, designed to enhance detection accuracy in the presence of input noise.<n>Our experiments on a modified CIC-IDS 2017 data set with synthetic noise demonstrate significant improvements in detection performance compared to the baseline TGN-SVDD model, especially as noise levels increase.
  arXiv  Detail & Related papers  (2025-08-19T18:36:11Z)
• GNN-ASE: Graph-Based Anomaly Detection and Severity Estimation in Three-Phase Induction Machines [0.0]
  This paper proposes a model-free approach using Graph Neural Networks (GNNs) for fault diagnosis in induction machines.<n>The proposed GNN-ASE model automatically learns and extracts relevant features from raw inputs.<n>It is evaluated for both individual fault detection and multi-class classification of combined fault conditions.
  arXiv  Detail & Related papers  (2025-07-23T01:50:07Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [78.18946529195254]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• Reliably Detecting Model Failures in Deployment Without Labels [14.331401006402979]
  This paper formalizes and addresses the problem of post-deployment deterioration (PDD) monitoring.<n>We propose D3M, a practical and efficient monitoring algorithm based on the disagreement of predictive models.<n> Empirical results on both standard benchmark and a real-world large-scale internal medicine dataset demonstrate the effectiveness of the framework.
  arXiv  Detail & Related papers  (2025-06-05T13:56:18Z)
• Counterfactual Explanation for Auto-Encoder Based Time-Series Anomaly Detection [0.3199881502576702]
  Auto-Encoders exhibit inherent opaqueness in their decision-making processes, hindering their practical implementation at scale.<n>In this work, we employ a feature selector to select features and counterfactual explanations to give a context to the model output.<n>Our experimental findings illustrate that our proposed counterfactual approach can offer meaningful and valuable insights into the model decision-making process.
  arXiv  Detail & Related papers  (2025-01-03T19:30:11Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Bisimulation metric for Model Predictive Control [44.301098448479195]
  Bisimulation Metric for Model Predictive Control (BS-MPC) is a novel approach that incorporates bisimulation metric loss in its objective function to directly optimize the encoder. BS-MPC improves training stability, robustness against input noise, and computational efficiency by reducing training time. We evaluate BS-MPC on both continuous control and image-based tasks from the DeepMind Control Suite.
  arXiv  Detail & Related papers  (2024-10-06T17:12:10Z)
• ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection [44.21198064126152]
  We propose a novel anomaly detection framework named ImDiffusion. ImDiffusion combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:57:40Z)
• On the benefits of robust models in modulation recognition [53.391095789289736]
  Deep Neural Networks (DNNs) using convolutional layers are state-of-the-art in many tasks in communications. In other domains, like image classification, DNNs have been shown to be vulnerable to adversarial perturbations. We propose a novel framework to test the robustness of current state-of-the-art models.
  arXiv  Detail & Related papers  (2021-03-27T19:58:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.