Large Scale Evaluation of Deep Learning-based Explainable Solar Flare Forecasting Models with Attribution-based Proximity Analysis

• URL: http://arxiv.org/abs/2411.18070v1
• Date: Wed, 27 Nov 2024 05:43:34 GMT
• Title: Large Scale Evaluation of Deep Learning-based Explainable Solar Flare Forecasting Models with Attribution-based Proximity Analysis
• Authors: Temitope Adeyeha, Chetraj Pandey, Berkay Aydin,
• Abstract summary: We propose a novel framework for assessing the interpretability of deep learning models for solar flare prediction. Our study compares two models trained on full-disk line-of-sight (LoS) magnetogram images to predict flares within a 24-hour window. Our findings indicate that the models' predictions align with active region characteristics to varying degrees, offering valuable insights into their behavior.
• Score: 0.0
• License:
• Abstract: Accurate and reliable predictions of solar flares are essential due to their potentially significant impact on Earth and space-based infrastructure. Although deep learning models have shown notable predictive capabilities in this domain, current evaluations often focus on accuracy while neglecting interpretability and reliability--factors that are especially critical in operational settings. To address this gap, we propose a novel proximity-based framework for analyzing post hoc explanations to assess the interpretability of deep learning models for solar flare prediction. Our study compares two models trained on full-disk line-of-sight (LoS) magnetogram images to predict $\geq$M-class solar flares within a 24-hour window. We employ the Guided Gradient-weighted Class Activation Mapping (Guided Grad-CAM) method to generate attribution maps from these models, which we then analyze to gain insights into their decision-making processes. To support the evaluation of explanations in operational systems, we introduce a proximity-based metric that quantitatively assesses the accuracy and relevance of local explanations when regions of interest are known. Our findings indicate that the models' predictions align with active region characteristics to varying degrees, offering valuable insights into their behavior. This framework enhances the evaluation of model interpretability in solar flare forecasting and supports the development of more transparent and reliable operational systems.

Related papers

• Towards Hybrid Embedded Feature Selection and Classification Approach with Slim-TSF [0.0]
  This study aims to uncover hidden relationships and the evolutionary characteristics of solar flares and their source regions. Preliminary findings indicate a notable improvement, with an average increase of 5% in both the True Skill Statistic (TSS) and Heidke Skill Score (HSS)
  arXiv  Detail & Related papers  (2024-09-06T18:12:05Z)
• A Bayesian Approach to Robust Inverse Reinforcement Learning [54.24816623644148]
  We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL) The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics. Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed to have a highly accurate model of the environment.
  arXiv  Detail & Related papers  (2023-09-15T17:37:09Z)
• Towards Interpretable Solar Flare Prediction with Attention-based Deep Neural Networks [1.1624569521079424]
  Solar flare prediction is a central problem in space weather forecasting. We developed an attention-based deep learning model to perform full-disk binary flare predictions. Our model can learn conspicuous features corresponding to active regions from full-disk magnetogram images.
  arXiv  Detail & Related papers  (2023-09-08T19:21:10Z)
• Exploring Deep Learning for Full-disk Solar Flare Prediction with Empirical Insights from Guided Grad-CAM Explanations [4.085931783551287]
  This study progresses solar flare prediction research by presenting a full-disk deep-learning model to forecast $geq$M-class solar flares. Our analysis unveils that full-disk solar flare predictions correspond with active region characteristics.
  arXiv  Detail & Related papers  (2023-08-30T02:24:09Z)
• Evaluating and Explaining Large Language Models for Code Using Syntactic Structures [74.93762031957883]
  This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes. We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
  arXiv  Detail & Related papers  (2023-08-07T18:50:57Z)
• Explainable Deep Learning-based Solar Flare Prediction with post hoc Attention for Operational Forecasting [0.6299766708197884]
  This paper presents a post hoc analysis of a deep learning-based full-disk solar flare prediction model. We used hourly full-disk line-of-sight magnetogram images and selected binary prediction mode to predict the occurrence of flares within 24 hours. Our analysis shows that full-disk predictions of solar flares align with characteristics related to the active regions.
  arXiv  Detail & Related papers  (2023-08-04T19:33:25Z)
• Explaining Full-disk Deep Learning Model for Solar Flare Prediction using Attribution Methods [0.6882042556551611]
  We present a solar flare prediction model, which is trained using hourly full-disk line-of-sight magnetogram images. We evaluate the overall performance of our model using the true skill statistic (TSS) and Heidke skill score (HSS) Our analysis revealed that full-disk prediction of solar flares aligns with characteristics related to active regions (ARs)
  arXiv  Detail & Related papers  (2023-07-29T03:18:56Z)
• A Comparative Study on Generative Models for High Resolution Solar Observation Imaging [59.372588316558826]
  This work investigates capabilities of current state-of-the-art generative models to accurately capture the data distribution behind observed solar activity states. Using distributed training on supercomputers, we are able to train generative models for up to 1024x1024 resolution that produce high quality samples indistinguishable to human experts.
  arXiv  Detail & Related papers  (2023-04-14T14:40:32Z)
• TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction [64.63645677568384]
  We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our code will be publicly available on GitHub.
  arXiv  Detail & Related papers  (2023-01-05T22:10:16Z)
• Spatial machine-learning model diagnostics: a model-agnostic distance-based approach [91.62936410696409]
  This contribution proposes spatial prediction error profiles (SPEPs) and spatial variable importance profiles (SVIPs) as novel model-agnostic assessment and interpretation tools. The SPEPs and SVIPs of geostatistical methods, linear models, random forest, and hybrid algorithms show striking differences and also relevant similarities. The novel diagnostic tools enrich the toolkit of spatial data science, and may improve ML model interpretation, selection, and design.
  arXiv  Detail & Related papers  (2021-11-13T01:50:36Z)
• Value-driven Hindsight Modelling [68.658900923595]
  Value estimation is a critical component of the reinforcement learning (RL) paradigm. Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function. We develop an approach for representation learning in RL that sits in between these two extremes. This provides tractable prediction targets that are directly relevant for a task, and can thus accelerate learning the value function.
  arXiv  Detail & Related papers  (2020-02-19T18:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.