CBAM-SwinT-BL: Small Rail Surface Defect Detection Method Based on Swin Transformer with Block Level CBAM Enhancement

• URL: http://arxiv.org/abs/2409.20113v2
• Date: Wed, 2 Oct 2024 07:16:05 GMT
• Title: CBAM-SwinT-BL: Small Rail Surface Defect Detection Method Based on Swin Transformer with Block Level CBAM Enhancement
• Abstract summary: This study utilizes Swin Transformer (SwinT) as baseline and incorporates the Convolutional Block Attention Module (CBAM) for enhancement. Our proposed method integrates CBAM within the swin transformer blocks, resulting in significant performance improvement in rail defect detection. Experiment and ablation study have proven the effectiveness of the framework.
• Score: 2.48550866840644
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Under high-intensity rail operations, rail tracks endure considerable stresses resulting in various defects such as corrugation and spellings. Failure to effectively detect defects and provide maintenance in time would compromise service reliability and public safety. While advanced models have been developed in recent years, efficiently identifying small-scale rail defects has not yet been studied, especially for categories such as Dirt or Squat on rail surface. To address this challenge, this study utilizes Swin Transformer (SwinT) as baseline and incorporates the Convolutional Block Attention Module (CBAM) for enhancement. Our proposed method integrates CBAM successively within the swin transformer blocks, resulting in significant performance improvement in rail defect detection, particularly for categories with small instance sizes. The proposed framework is named CBAM-Enhanced Swin Transformer in Block Level (CBAM-SwinT-BL). Experiment and ablation study have proven the effectiveness of the framework. The proposed framework has a notable improvement in the accuracy of small size defects, such as dirt and dent categories in RIII dataset, with mAP-50 increasing by +23.0% and +38.3% respectively, and the squat category in MUET dataset also reaches +13.2% higher than the original model. Compares to the original SwinT, CBAM-SwinT-BL increase overall precision around +5% in the MUET dataset and +7% in the RIII dataset, reaching 69.1% and 88.1% respectively. Meanwhile, the additional module CBAM merely extend the model training speed by an average of +0.04s/iteration, which is acceptable compared to the significant improvement in system performance.

Related papers

• Test-time Adaptation of Pelvic Bone Segmentation Models via Dynamic Reliability-Guided [62.10276009853225]
  Reliable pelvic bone segmentation from CT is essential for robot-assisted pelvic trauma surgery.<n>Existing methods show limited effectiveness for PBS.<n>We propose a novel closed-loop dynamic Reliability-Guided framework (ReGA) for PBS.
  arXiv  Detail & Related papers  (2026-08-01T08:13:20Z)
• How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap [0.0]
  Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable.<n>We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwiseseparable convolutional U-Net.
  arXiv  Detail & Related papers  (2026-06-07T12:17:25Z)
• Novel Dynamic Batch-Sensitive Adam Optimiser for Vehicular Accident Injury Severity Prediction [0.0]
  We propose Dynamic Batch-Sensitive Adam (DBS-Adam) as an optimiser for learning from imbalanced datasets.<n>DBS-Adam dynamically scales the learning rate using a batch difficulty score derived from exponential moving averages of gradient norms and batch loss.<n>It achieves 95.22% test accuracy, 96.11% precision, 95.28% recall, 95.39% F1-score, and a test loss of 0.0086.
  arXiv  Detail & Related papers  (2026-05-14T17:06:43Z)
• Feature-Augmented Transformers for Robust AI-Text Detection Across Domains and Generators [0.10923877073891443]
  We train transformer-based detectors on HC3 PLUS and calibrate a single decision threshold by maximising balanced accuracy on held-out validation.<n>We evaluate in-domain on HC3 PLUS, under cross-dataset transfer to the multi-domain, multi-generator M4 benchmark, and on the external AI-Text-Detection-Pile.<n>Our best model (DeBERTa-v3-base+FeatAttn) achieves 85.9% balanced accuracy on M4.
  arXiv  Detail & Related papers  (2026-05-05T16:52:26Z)
• TEMPO: Scaling Test-time Training for Large Reasoning Models [87.61789183311856]
  Test-time training (TTT) adapts model parameters on unlabeled test instances during inference time.<n>We propose TEMPO, a TTT framework that interleaves policy refinement on unlabeled questions with periodic critic recalibration on a labeled dataset.
  arXiv  Detail & Related papers  (2026-04-21T10:01:04Z)
• Breaking the Data Barrier: Robust Few-Shot 3D Vessel Segmentation using Foundation Models [0.0]
  State-of-the-art vessel segmentation methods typically require large-scale annotated datasets and suffer from severe performance degradation under domain shifts.<n>We propose a novel framework leveraging a pre-trained Vision Foundation Model (DINOv3) adapted for volumetric vessel segmentation.
  arXiv  Detail & Related papers  (2026-02-27T08:24:44Z)
• CLEANet: Robust and Efficient Anomaly Detection in Contaminated Multivariate Time Series [17.992163690121565]
  CLEANet is a robust and efficient anomaly detection framework in contaminated time series.<n>Cleanet introduces a Contamination-Resilient Training Framework (CRTF)<n>Cleanet mitigates higher F1 and 81.28% lower runtime compared with ten state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-10-26T10:34:19Z)
• Multi-Channel Swin Transformer Framework for Bearing Remaining Useful Life Prediction [2.600463444320238]
  This paper introduces a novel framework that combines wavelet-based denoising method, Wavelet Packet Decomposition (WPD), and a customized multi-channel Swin Transformer model (MCSFormer) to address these problems.<n>With attention mechanisms incorporated for feature fusion, the model is designed to learn global and local degradation patterns.<n>A customized loss function is developed as a key distinction of this work to differentiate between early and late predictions.
  arXiv  Detail & Related papers  (2025-05-20T20:44:38Z)
• Semiconductor Wafer Map Defect Classification with Tiny Vision Transformers [0.0]
  We propose ViT-Tiny, a lightweight Vision Transformer (ViT) framework optimized for wafer defect classification. ViT-Tiny outperforms its ViT-Base counterpart and state-of-the-art (SOTA) models, such as MSF-Trans and CNN-based architectures. It achieves an F1-score of 98.4%, surpassing MSF-Trans by 2.94% in four-defect classification, improving recall by 2.86% in two-defect classification, and increasing precision by 3.13% in three-defect classification.
  arXiv  Detail & Related papers  (2025-04-03T11:18:00Z)
• Transfer Learning with Active Sampling for Rapid Training and Calibration in BCI-P300 Across Health States and Multi-centre Data [0.13124513975412253]
  Machine learning and deep learning advancements have boosted Brain-Computer Interface (BCI) performance. Their wide-scale applicability is limited due to factors like individual health, hardware variations, and cultural differences affecting neural data. We propose P300 wave detection in BCIs employing a convolutional neural network fitted with adaptive transfer learning.
  arXiv  Detail & Related papers  (2024-12-14T14:20:21Z)
• Efficient Training of Transformers for Molecule Property Prediction on Small-scale Datasets [0.0]
  Blood-brain barrier (BBB) serves as a protective barrier that separates the brain from the circulatory system. Traditional experimental methods for measuring BBB permeability are challenging and impractical for large-scale screening. This paper proposes a GPS Transformer architecture augmented with Self Attention, designed to perform well in the low-data regime.
  arXiv  Detail & Related papers  (2024-09-07T21:07:12Z)
• BEA: Revisiting anchor-based object detection DNN using Budding Ensemble Architecture [8.736601342033431]
  Budding Ensemble Architecture (BEA) is a novel reduced ensemble architecture for anchor-based object detection models. The proposed loss functions in BEA improve the confidence score calibration and lower the uncertainty error.
  arXiv  Detail & Related papers  (2023-09-14T21:54:23Z)
• Patch-Level Contrasting without Patch Correspondence for Accurate and Dense Contrastive Representation Learning [79.43940012723539]
  ADCLR is a self-supervised learning framework for learning accurate and dense vision representation. Our approach achieves new state-of-the-art performance for contrastive methods.
  arXiv  Detail & Related papers  (2023-06-23T07:38:09Z)
• Stable and Efficient Adversarial Training through Local Linearization [0.5076419064097734]
  A phenomenon referred to as catastrophic overfitting" has been observed, which is prevalent in single-step defenses. We propose a novel method, Stable and Efficient Adversarial Training (SEAT), which mitigates catastrophic overfitting. Our single-step method can reach 51% robust accuracy for CIFAR-10 with $l_infty$ perturbations of radius $8/255$ under a strong PGD-50 attack.
  arXiv  Detail & Related papers  (2022-10-11T11:57:37Z)
• ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention [48.697458429460184]
  Two factors, information bottleneck sensitivity and inconsistency between different attention topologies, could affect the performance of the Sparse Transformer. This paper proposes a well-designed model named ERNIE-Sparse. It consists of two distinctive parts: (i) Hierarchical Sparse Transformer (HST) to sequentially unify local and global information, and (ii) Self-Attention Regularization (SAR) to minimize the distance for transformers with different attention topologies.
  arXiv  Detail & Related papers  (2022-03-23T08:47:01Z)
• Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation [122.83280749890078]
  We propose an improved certified defense against general poisoning attacks, namely Finite Aggregation. In contrast to DPA, which directly splits the training set into disjoint subsets, our method first splits the training set into smaller disjoint subsets. We offer an alternative view of our method, bridging the designs of deterministic and aggregation-based certified defenses.
  arXiv  Detail & Related papers  (2022-02-05T20:08:58Z)
• Fast Convergence of DETR with Spatially Modulated Co-Attention [83.19863907905666]
  We propose a simple yet effective scheme for improving the Detection Transformer framework, namely Spatially Modulated Co-Attention (SMCA) mechanism. Our proposed SMCA increases DETR's convergence speed by replacing the original co-attention mechanism in the decoder. Our fully-fledged SMCA can achieve better performance compared to DETR with a dilated convolution-based backbone.
  arXiv  Detail & Related papers  (2021-08-05T06:53:19Z)
• Fast Convergence of DETR with Spatially Modulated Co-Attention [83.19863907905666]
  We propose a simple yet effective scheme for improving the Detection Transformer framework, namely Spatially Modulated Co-Attention (SMCA) mechanism. Our proposed SMCA increases DETR's convergence speed by replacing the original co-attention mechanism in the decoder. Our fully-fledged SMCA can achieve better performance compared to DETR with a dilated convolution-based backbone.
  arXiv  Detail & Related papers  (2021-01-19T03:52:44Z)
• Inception Convolution with Efficient Dilation Search [121.41030859447487]
  Dilation convolution is a critical mutant of standard convolution neural network to control effective receptive fields and handle large scale variance of objects. We propose a new mutant of dilated convolution, namely inception (dilated) convolution where the convolutions have independent dilation among different axes, channels and layers. We explore a practical method for fitting the complex inception convolution to the data, a simple while effective dilation search algorithm(EDO) based on statistical optimization is developed.
  arXiv  Detail & Related papers  (2020-12-25T14:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.