A Machine-Learning-Ready Dataset Prepared from the Solar and Heliospheric Observatory Mission

• URL: http://arxiv.org/abs/2108.06394v1
• Date: Wed, 4 Aug 2021 21:23:29 GMT
• Title: A Machine-Learning-Ready Dataset Prepared from the Solar and Heliospheric Observatory Mission
• Authors: Carl Shneider (1), Andong Hu (1), Ajay K. Tiwari (1), Monica G. Bobra (2), Karl Battams (5), Jannis Teunissen (1), and Enrico Camporeale (3 and 4) ((1) Multiscale Dynamics Group, Center for Mathematics and Computer Science (CWI), Amsterdam, The Netherlands, (2) W.W. Hansen Experimental Physics Laboratory, Stanford University, Stanford, CA, USA, (3) CIRES, University of Colorado, Boulder, CO, USA, (4) NOAA, Space Weather Prediction Center, Boulder, CO, USA, (5) US Naval Research Laboratory, Washington DC, USA)
• Abstract summary: We present a Python tool to generate a standard dataset from solar images. Our tool works with all image products from both the Solar and Heliospheric Observatory (SoHO) and Solar Dynamics Observatory (SDO) missions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a Python tool to generate a standard dataset from solar images that allows for user-defined selection criteria and a range of pre-processing steps. Our Python tool works with all image products from both the Solar and Heliospheric Observatory (SoHO) and Solar Dynamics Observatory (SDO) missions. We discuss a dataset produced from the SoHO mission's multi-spectral images which is free of missing or corrupt data as well as planetary transits in coronagraph images, and is temporally synced making it ready for input to a machine learning system. Machine-learning-ready images are a valuable resource for the community because they can be used, for example, for forecasting space weather parameters. We illustrate the use of this data with a 3-5 day-ahead forecast of the north-south component of the interplanetary magnetic field (IMF) observed at Lagrange point one (L1). For this use case, we apply a deep convolutional neural network (CNN) to a subset of the full SoHO dataset and compare with baseline results from a Gaussian Naive Bayes classifier.

Related papers

• TelescopeML -- I. An End-to-End Python Package for Interpreting Telescope Datasets through Training Machine Learning Models, Generating Statistical Reports, and Visualizing Results [1.3372051498158442]
  textttTelescopeML is a Python package developed to perform three main tasks. Process the synthetic astronomical datasets for training a CNN model and prepare the observational dataset for later use for prediction.
  arXiv  Detail & Related papers  (2024-07-24T00:44:52Z)
• M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data [1.4053129774629076]
  M3LEO is a multi-modal, multi-label Earth observation dataset. It spans approximately 17M 4x4 km data chips from six diverse geographic regions.
  arXiv  Detail & Related papers  (2024-06-06T16:30:41Z)
• Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery [78.43828998065071]
  Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks. Such pre-training techniques have also been explored recently in the remote sensing domain due to the availability of large amount of unlabelled data. In this paper, we re-visit transformers pre-training and leverage multi-scale information that is effectively utilized with multiple modalities.
  arXiv  Detail & Related papers  (2024-03-08T16:18:04Z)
• Solar Active Region Magnetogram Image Dataset for Studies of Space Weather [0.0]
  The dataset incorporates data from three sources and provides SDO Helioseismic and Magnetic Imager (HMI) magnetograms of solar active regions. This dataset will be useful for image analysis or solar physics research related to magnetic structure, its evolution over time, and its relation to solar flares. This dataset is a minimally processed, user dataset of consistently sized images of solar active regions that can serve as a benchmark dataset for solar flare prediction research.
  arXiv  Detail & Related papers  (2023-05-16T14:44:24Z)
• Semantic Segmentation of Vegetation in Remote Sensing Imagery Using Deep Learning [77.34726150561087]
  We propose an approach for creating a multi-modal and large-temporal dataset comprised of publicly available Remote Sensing data. We use Convolutional Neural Networks (CNN) models that are capable of separating different classes of vegetation.
  arXiv  Detail & Related papers  (2022-09-28T18:51:59Z)
• Satellite Image Time Series Analysis for Big Earth Observation Data [50.591267188664666]
  This paper describes sits, an open-source R package for satellite image time series analysis using machine learning. We show that this approach produces high accuracy for land use and land cover maps through a case study in the Cerrado biome.
  arXiv  Detail & Related papers  (2022-04-24T15:23:25Z)
• A Dataset of Images of Public Streetlights with Operational Monitoring using Computer Vision Techniques [56.838577982762956]
  dataset consists of $sim350textrmk$ images, taken from 140 UMBRELLA nodes installed in the South Gloucestershire region in the UK. The dataset can be used to train deep neural networks and generate pre-trained models for smart city CCTV applications, smart weather detection algorithms, or street infrastructure monitoring.
  arXiv  Detail & Related papers  (2022-03-31T09:36:07Z)
• Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans [103.92680099373567]
  This paper introduces a pipeline to parametrically sample and render multi-task vision datasets from comprehensive 3D scans from the real world. Changing the sampling parameters allows one to "steer" the generated datasets to emphasize specific information. Common architectures trained on a generated starter dataset reached state-of-the-art performance on multiple common vision tasks and benchmarks.
  arXiv  Detail & Related papers  (2021-10-11T04:21:46Z)
• DeepSatData: Building large scale datasets of satellite images for training machine learning models [77.17638664503215]
  This report presents design considerations for automatically generating satellite imagery datasets for training machine learning models. We discuss issues faced from the point of view of deep neural network training and evaluation.
  arXiv  Detail & Related papers  (2021-04-28T15:13:12Z)
• Identifying and Tracking Solar Magnetic Flux Elements with Deep Learning [6.659099851744079]
  We propose a new deep learning method, called SolarUnet, to identify and track solar magnetic flux elements or features in observed vector magnetograms. SolarUnet is applied to data from the 1.6 meter Goode Solar Telescope at the Big Bear Solar Observatory.
  arXiv  Detail & Related papers  (2020-08-27T12:23:18Z)
• Self-supervised Learning for Astronomical Image Classification [1.2891210250935146]
  In Astronomy, a huge amount of image data is generated daily by photometric surveys. We propose a technique to leverage unlabeled astronomical images to pre-train deep convolutional neural networks.
  arXiv  Detail & Related papers  (2020-04-23T17:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.