MEMPSEP III. A machine learning-oriented multivariate data set for forecasting the Occurrence and Properties of Solar Energetic Particle Events using a Multivariate Ensemble Approach

• URL: http://arxiv.org/abs/2310.15390v2
• Date: Thu, 26 Oct 2023 20:48:52 GMT
• Title: MEMPSEP III. A machine learning-oriented multivariate data set for forecasting the Occurrence and Properties of Solar Energetic Particle Events using a Multivariate Ensemble Approach
• Abstract summary: This paper describes a dataset created from multiple publicly available observation sources that is validated, cleaned, and curated for our machine-learning pipeline. We identify 252 solar events (flares) that produce solar energetic particles (SEPs) and 17,542 events that do not. For each identified event, we acquire the local plasma properties at 1 au, such as energetic proton and electron data, upstream solar wind conditions, and the interplanetary magnetic field vector quantities.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a new multivariate data set that utilizes multiple spacecraft collecting in-situ and remote sensing heliospheric measurements shown to be linked to physical processes responsible for generating solar energetic particles (SEPs). Using the Geostationary Operational Environmental Satellites (GOES) flare event list from Solar Cycle (SC) 23 and part of SC 24 (1998-2013), we identify 252 solar events (flares) that produce SEPs and 17,542 events that do not. For each identified event, we acquire the local plasma properties at 1 au, such as energetic proton and electron data, upstream solar wind conditions, and the interplanetary magnetic field vector quantities using various instruments onboard GOES and the Advanced Composition Explorer (ACE) spacecraft. We also collect remote sensing data from instruments onboard the Solar Dynamic Observatory (SDO), Solar and Heliospheric Observatory (SoHO), and the Wind solar radio instrument WAVES. The data set is designed to allow for variations of the inputs and feature sets for machine learning (ML) in heliophysics and has a specific purpose for forecasting the occurrence of SEP events and their subsequent properties. This paper describes a dataset created from multiple publicly available observation sources that is validated, cleaned, and carefully curated for our machine-learning pipeline. The dataset has been used to drive the newly-developed Multivariate Ensemble of Models for Probabilistic Forecast of Solar Energetic Particles (MEMPSEP; see MEMPSEP I (Chatterjee et al., 2023) and MEMPSEP II (Dayeh et al., 2023) for associated papers).

Related papers

• Review of Machine Learning Models for Solar Energetic Particle Prediction [28.052312775333615]
  Solar energetic particle (SEP) events have attracted increasing attention due to their significant radiation hazards for aviation, spacecraft electronics, and human missions beyond Earth's magnetosphere.<n>Machine learning (ML) has emerged as a new tool for understanding and predicting SEP events.<n>The purpose of this manuscript is to review the currently available ML models for SEP prediction, identify the datasets used for training, compare their architectures, inputs, and outputs, and, based on these insights, outline good practices and recommendations for future research.
  arXiv  Detail & Related papers  (2026-06-17T19:30:27Z)
• Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting [52.53742727060265]
  Foundation models (FMs) for the Earth system learn statistical relationships between physical variables across massive datasets.<n>Here, we introduce Earth System Foundation Model (ESFM), a fully open model building on the 3Dwin UNet backbone of the pioneering model.<n>We present findings using dense gridded data (ERA5, CMIP6), regionally masked dense data, sparse gridded MODIS satellite data, and station data.<n>Results demonstrate competitive or superior performance relative to state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T11:40:39Z)
• Enhancing Explainability in Solar Energetic Particle Event Prediction: A Global Feature Mapping Approach [0.0]
  Solar energetic particle (SEP) events can generate hazardous radiation when accelerated by solar flares or shock waves formed aside from coronal mass ejections (CMEs)<n>Most existing data-driven methods used for SEP predictions are operated as black-box models.<n>We propose a novel framework that integrates global explanations and ad-hoc feature mapping to enhance model transparency.
  arXiv  Detail & Related papers  (2025-11-12T16:47:36Z)
• Scalable Machine Learning Analysis of Parker Solar Probe Solar Wind Data [0.0]
  We present a scalable machine learning framework for analyzing Parker Solar Probe (PSP) solar wind data.<n>The PSP dataset exceeds 150 GB, challenging conventional analysis approaches.<n>We reveal characteristic trends in the inner heliosphere, including increasing solar wind speed with distance from the Sun.
  arXiv  Detail & Related papers  (2025-10-24T00:41:39Z)
• Ultra-short-term solar power forecasting by deep learning and data reconstruction [60.200987006598524]
  We propose a deep-learning based ultra-short-term solar power prediction with data reconstruction.<n>We employ deep-learning models to capture long- and short-term dependencies towards the target prediction period.
  arXiv  Detail & Related papers  (2025-09-21T14:22:35Z)
• SuryaBench: Benchmark Dataset for Advancing Machine Learning in Heliophysics and Space Weather Prediction [2.288747975391298]
  This paper introduces a high resolution, machine learning-ready heliophysics dataset derived from NASA's Solar Dynamics Observatory (SDO)<n>The dataset includes processed imagery from the Atmospheric Imaging Assembly (AIA) and Helioseismic and Magnetic Imager (HMI)<n>To ensure suitability for ML tasks, the data has been preprocessed, including correction of spacecraft roll angles, orbital adjustments, exposure normalization, and degradation compensation.
  arXiv  Detail & Related papers  (2025-08-18T00:05:01Z)
• Conceptual framework for the application of deep neural networks to surface composition reconstruction from Mercury's exospheric data [77.40388962445168]
  This study explores the feasibility of deriving Mercury's regolith elemental composition from in-situ measurements of its neutral exosphere using deep neural networks (DNNs)<n>We present a supervised feed-forward DNN architecture that predicts the chemical elements of the surface regolith below.<n>It serves as an estimator for the surface-exosphere interaction and the processes leading to exosphere formation.
  arXiv  Detail & Related papers  (2025-05-16T09:52:45Z)
• PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model [76.95536611263356]
  PolSAR data presents unique challenges due to its rich and complex characteristics. Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used. Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively. We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy.
  arXiv  Detail & Related papers  (2024-12-17T09:59:53Z)
• Solar synthetic imaging: Introducing denoising diffusion probabilistic models on SDO/AIA data [0.0]
  This study proposes using generative deep learning models, specifically a Denoising Diffusion Probabilistic Model (DDPM), to create synthetic images of solar phenomena. By employing a dataset from the AIA instrument aboard the SDO spacecraft, we aim to address the data scarcity issue. The DDPM's performance is evaluated using cluster metrics, Frechet Inception Distance (FID), and F1-score, showcasing promising results in generating realistic solar imagery.
  arXiv  Detail & Related papers  (2024-04-03T08:18:45Z)
• Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation [48.66623377464203]
  Our novel approach introduces the Dynamic One-For-All (DOFA) model, leveraging the concept of neural plasticity in brain science. This dynamic hypernetwork, adjusting to different wavelengths, enables a single versatile Transformer jointly trained on data from five sensors to excel across 12 distinct Earth observation tasks.
  arXiv  Detail & Related papers  (2024-03-22T17:11:47Z)
• Forecasting SEP Events During Solar Cycles 23 and 24 Using Interpretable Machine Learning [38.321248253111776]
  We employ a suite of machine learning strategies to evaluate the predictive potential of a new data product for a forecast of post-solar flare SEP events. Despite the augmented volume of data, the prediction accuracy reaches 0.7 +- 0.1, which aligns with but does not exceed these published benchmarks.
  arXiv  Detail & Related papers  (2024-03-04T23:12:17Z)
• Multimodal Dataset from Harsh Sub-Terranean Environment with Aerosol Particles for Frontier Exploration [55.41644538483948]
  This paper introduces a multimodal dataset from the harsh and unstructured underground environment with aerosol particles. It contains synchronized raw data measurements from all onboard sensors in Robot Operating System (ROS) format. The focus of this paper is not only to capture both temporal and spatial data diversities but also to present the impact of harsh conditions on captured data.
  arXiv  Detail & Related papers  (2023-04-27T20:21:18Z)
• A Comparative Study on Generative Models for High Resolution Solar Observation Imaging [59.372588316558826]
  This work investigates capabilities of current state-of-the-art generative models to accurately capture the data distribution behind observed solar activity states. Using distributed training on supercomputers, we are able to train generative models for up to 1024x1024 resolution that produce high quality samples indistinguishable to human experts.
  arXiv  Detail & Related papers  (2023-04-14T14:40:32Z)
• Computational Solar Energy -- Ensemble Learning Methods for Prediction of Solar Power Generation based on Meteorological Parameters in Eastern India [0.0]
  It is important to estimate the amount of solar photovoltaic (PV) power generation for a specific geographical location. In this paper, the impact of weather parameters on solar PV power generation is estimated by several Ensemble ML (EML) models like Bagging, Boosting, Stacking, and Voting. The results demonstrate greater prediction accuracy of around 96% for Stacking and Voting models.
  arXiv  Detail & Related papers  (2023-01-21T19:16:03Z)
• Machine learning model to predict solar radiation, based on the integration of meteorological data and data obtained from satellite images [0.0]
  Knowing the behavior of solar radiation at a geographic location is essential for the use of energy from the sun. Images obtained from the GOES-13 satellite were used, from which variables were extracted that could be integrated into datasets. The performance of 5 machine learning algorithms in predicting solar radiation was evaluated.
  arXiv  Detail & Related papers  (2022-04-08T22:17:19Z)
• Predicting Solar Energetic Particles Using SDO/HMI Vector Magnetic Data Products and a Bidirectional LSTM Network [6.759687230043489]
  Solar energetic particles (SEPs) are an essential source of space radiation, which are hazards for humans in space, spacecraft, and technology in general. We propose a deep learning method to predict if an active region (AR) would produce an SEP event given that (i) the AR will produce an M- or X-class flare and a coronal mass ejection associated with the flare, or (ii) the AR will produce an M- or X-class flare regardless of whether or not the flare is associated with a CME.
  arXiv  Detail & Related papers  (2022-03-27T21:06:08Z)
• DeepClimGAN: A High-Resolution Climate Data Generator [60.59639064716545]
  Earth system models (ESMs) are often used to generate future projections of climate change scenarios. As a compromise, emulators are substantially less expensive but may not have all of the complexity of an ESM. Here we demonstrate the use of a conditional generative adversarial network (GAN) to act as an ESM emulator.
  arXiv  Detail & Related papers  (2020-11-23T20:13:37Z)
• Energy Aware Deep Reinforcement Learning Scheduling for Sensors Correlated in Time and Space [62.39318039798564]
  We propose a scheduling mechanism capable of taking advantage of correlated information. The proposed mechanism is capable of determining the frequency with which sensors should transmit their updates. We show that our solution can significantly extend the sensors' lifetime.
  arXiv  Detail & Related papers  (2020-11-19T09:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.