Benchmarking Vision Transformers and CNNs for Thermal Photovoltaic Fault Detection with Explainable AI Validation

• URL: http://arxiv.org/abs/2509.07039v1
• Date: Mon, 08 Sep 2025 08:38:53 GMT
• Title: Benchmarking Vision Transformers and CNNs for Thermal Photovoltaic Fault Detection with Explainable AI Validation
• Authors: Serra Aksoy,
• Abstract summary: This study provides a systematic comparison of convolutional neural networks (ResNet-18, EfficientNet-B0) and vision transformers (ViT-Tiny, Swin-Tiny) for thermal PV fault detection.<n> Evaluation on 20,000 infrared images spanning normal operation and 11 fault categories shows that Swin Transformer achieves the highest performance.<n> XRAI analysis reveals that models learn physically meaningful features, such as localized hotspots for cell defects, linear thermal paths for diode failures, and thermal boundaries for vegetation shading.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence deployment for automated photovoltaic (PV) monitoring faces interpretability barriers that limit adoption in energy infrastructure applications. While deep learning achieves high accuracy in thermal fault detection, validation that model decisions align with thermal physics principles remains lacking, creating deployment hesitancy where understanding model reasoning is critical. This study provides a systematic comparison of convolutional neural networks (ResNet-18, EfficientNet-B0) and vision transformers (ViT-Tiny, Swin-Tiny) for thermal PV fault detection, using XRAI saliency analysis to assess alignment with thermal physics principles. This represents the first systematic comparison of CNNs and vision transformers for thermal PV fault detection with physics-validated interpretability. Evaluation on 20,000 infrared images spanning normal operation and 11 fault categories shows that Swin Transformer achieves the highest performance (94% binary accuracy; 73% multiclass accuracy) compared to CNN approaches. XRAI analysis reveals that models learn physically meaningful features, such as localized hotspots for cell defects, linear thermal paths for diode failures, and thermal boundaries for vegetation shading, consistent with expected thermal signatures. However, performance varies significantly across fault types: electrical faults achieve strong detection (F1-scores >0.90) while environmental factors like soiling remain challenging (F1-scores 0.20-0.33), indicating limitations imposed by thermal imaging resolution. The thermal physics-guided interpretability approach provides methodology for validating AI decision-making in energy monitoring applications, addressing deployment barriers in renewable energy infrastructure.

Related papers

• Application of Graph Based Vision Transformers Architectures for Accurate Temperature Prediction in Fiber Specklegram Sensors [0.0]
  This study investigates the use of transformer-based architectures to predict temperature from specklegram data over a range of 0 to 120 Celsius.<n>The results show that ViTs achieved a Mean Absolute Error (MAE) of 1.15, outperforming traditional models such as CNNs.<n>The study incorporates Explainable AI (XAI) techniques, including attention maps and saliency maps, to provide insights into the decision-making processes of the transformer models.
  arXiv  Detail & Related papers  (2025-11-15T07:56:15Z)
• HOTSPOT-YOLO: A Lightweight Deep Learning Attention-Driven Model for Detecting Thermal Anomalies in Drone-Based Solar Photovoltaic Inspections [0.685316573653194]
  HotSPOT-YOLO is a lightweight artificial intelligence (AI) model that integrates an efficient convolutional neural network backbone and attention mechanisms to improve object detection.<n>This work highlights the integration of advanced AI techniques with practical engineering applications, revolutionizing automated fault detection in renewable energy systems.
  arXiv  Detail & Related papers  (2025-08-26T10:35:24Z)
• Data-driven Power Loss Identification through Physics-Based Thermal Model Backpropagation [1.9997803560872798]
  Digital twins for power electronics require accurate power losses whose direct measurements are often impractical or impossible in real-world applications.<n>This paper presents a novel hybrid framework that combines physics-based thermal modeling with data-driven techniques to identify and correct power losses accurately using only temperature measurements.
  arXiv  Detail & Related papers  (2025-03-31T18:37:14Z)
• Optical aberrations in autonomous driving: Physics-informed parameterized temperature scaling for neural network uncertainty calibration [49.03824084306578]
  We propose to incorporate a physical inductive bias into the neural network calibration architecture to enhance the robustness and the trustworthiness of the AI target application.<n>We pave the way for a trustworthy uncertainty representation and for a holistic verification strategy of the perception chain.
  arXiv  Detail & Related papers  (2024-12-18T10:36:46Z)
• AI-Powered Dynamic Fault Detection and Performance Assessment in Photovoltaic Systems [44.99833362998488]
  intermittent nature of photovoltaic (PV) solar energy leads to power losses of 10-70% and an average energy production decrease of 25%. Current fault detection strategies are costly and often yield unreliable results due to complex data signal profiles. This research presents a computational model using the PVlib library in Python, incorporating a dynamic loss quantification algorithm.
  arXiv  Detail & Related papers  (2024-08-19T23:52:06Z)
• Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants [0.6223528900192875]
  This article introduces a-temporal model for transformer winding temperature and ageing. It uses physics-based partial differential equations with data-driven Neural Networks. Results are validated with a distribution transformer operating on a floating photovoltaic power plant.
  arXiv  Detail & Related papers  (2024-05-10T12:48:57Z)
• Ranking-Based Physics-Informed Line Failure Detection in Power Grids [66.0797334582536]
  Real-time and accurate detecting of potential line failures is the first step to mitigating the extreme weather impact and activating emergency controls. Power balance equations nonlinearity, increased uncertainty in generation during extreme events, and lack of grid observability compromise the efficiency of traditional data-driven failure detection methods. This paper proposes a Physics-InformEd Line failure Detector (FIELD) that leverages grid topology information to reduce sample and time complexities and improve localization accuracy.
  arXiv  Detail & Related papers  (2022-08-31T18:19:25Z)
• Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction [51.80191416661064]
  We propose a novel vision transformer with latent variables following an informative energy-based prior for salient object detection. Both the vision transformer network and the energy-based prior model are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation. With the generative vision transformer, we can easily obtain a pixel-wise uncertainty map from an image, which indicates the model confidence in predicting saliency from the image.
  arXiv  Detail & Related papers  (2021-12-27T06:04:33Z)
• A Multi-Stage model based on YOLOv3 for defect detection in PV panels based on IR and Visible Imaging by Unmanned Aerial Vehicle [65.99880594435643]
  We propose a novel model to detect panel defects on aerial images captured by unmanned aerial vehicle. The model combines detections of panels and defects to refine its accuracy. The proposed model has been validated on two big PV plants in the south of Italy.
  arXiv  Detail & Related papers  (2021-11-23T08:04:32Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.