On-Demand Earth System Data Cubes

• URL: http://arxiv.org/abs/2404.13105v1
• Date: Fri, 19 Apr 2024 13:50:30 GMT
• Title: On-Demand Earth System Data Cubes
• Authors: David Montero, César Aybar, Chaonan Ji, Guido Kraemer, Maximilian Söchting, Khalil Teber, Miguel D. Mahecha,
• Abstract summary: ESDCs offer a structured, intuitive framework for data analysis. ESDCs are ideally suited for a wide range of AI-driven tasks. We introduce cubo, an open-source Python tool designed for easy generation of AI-focused ESDCs.
• Score: 2.062646422366945
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advancements in Earth system science have seen a surge in diverse datasets. Earth System Data Cubes (ESDCs) have been introduced to efficiently handle this influx of high-dimensional data. ESDCs offer a structured, intuitive framework for data analysis, organising information within spatio-temporal grids. The structured nature of ESDCs unlocks significant opportunities for Artificial Intelligence (AI) applications. By providing well-organised data, ESDCs are ideally suited for a wide range of sophisticated AI-driven tasks. An automated framework for creating AI-focused ESDCs with minimal user input could significantly accelerate the generation of task-specific training data. Here we introduce cubo, an open-source Python tool designed for easy generation of AI-focused ESDCs. Utilising collections in SpatioTemporal Asset Catalogs (STAC) that are stored as Cloud Optimised GeoTIFFs (COGs), cubo efficiently creates ESDCs, requiring only central coordinates, spatial resolution, edge size, and time range.

Related papers

• Are We Ready for Real-Time LiDAR Semantic Segmentation in Autonomous Driving? [42.348499880894686]
  Scene semantic segmentation can be achieved by directly integrating 3D spatial data with specialized deep neural networks. We investigate various 3D semantic segmentation methodologies and analyze their performance and capabilities for resource-constrained inference on embedded NVIDIA Jetson platforms.
  arXiv  Detail & Related papers  (2024-10-10T20:47:33Z)
• Earth System Data Cubes: Avenues for advancing Earth system research [4.408949931570938]
  Earth System Data Cubes ( ESDCs) have emerged as one suitable solution for transforming this flood of data into a simple yet robust format. ESDCs achieve this by organising data into an analysis-ready format with atemporal grid. There exist barriers to realising the full potential of data in light of novel cloud-based technologies.
  arXiv  Detail & Related papers  (2024-08-05T09:50:16Z)
• TSTEM: A Cognitive Platform for Collecting Cyber Threat Intelligence in the Wild [0.06597195879147556]
  The extraction of cyber threat intelligence (CTI) from open sources is a rapidly expanding defensive strategy. Previous research has focused on improving individual components of the extraction process. The community lacks open-source platforms for deploying streaming CTI data pipelines in the wild.
  arXiv  Detail & Related papers  (2024-02-15T14:29:21Z)
• CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning [62.58375643251612]
  We propose a Curiosity-driven Unsupervised Data Collection (CUDC) method to expand feature space using adaptive temporal distances for task-agnostic data collection. With this adaptive reachability mechanism in place, the feature representation can be diversified, and the agent can navigate itself to collect higher-quality data with curiosity. Empirically, CUDC surpasses existing unsupervised methods in efficiency and learning performance in various downstream offline RL tasks of the DeepMind control suite.
  arXiv  Detail & Related papers  (2023-12-19T14:26:23Z)
• The GOOSE Dataset for Perception in Unstructured Environments [3.0408645115035036]
  We present a comprehensive dataset specifically designed for unstructured outdoor environments. The GOOSE dataset incorporates 10 000 labeled pairs of images and point clouds, which are utilized to train a range of state-of-the-art segmentation models. This initiative aims to establish a common framework, enabling the seamless inclusion of existing datasets and a fast way to enhance the perception capabilities of various robots operating in unstructured environments.
  arXiv  Detail & Related papers  (2023-10-25T17:20:38Z)
• AutoSynth: Learning to Generate 3D Training Data for Object Point Cloud Registration [69.21282992341007]
  Auto Synth automatically generates 3D training data for point cloud registration. We replace the point cloud registration network with a much smaller surrogate network, leading to a $4056.43$ speedup. Our results on TUD-L, LINEMOD and Occluded-LINEMOD evidence that a neural network trained on our searched dataset yields consistently better performance than the same one trained on the widely used ModelNet40 dataset.
  arXiv  Detail & Related papers  (2023-09-20T09:29:44Z)
• Data-centric Artificial Intelligence: A Survey [47.24049907785989]
  Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle.
  arXiv  Detail & Related papers  (2023-03-17T17:44:56Z)
• DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems [81.21462458089142]
  Data-centric AI is emerging as a unifying paradigm that could enable reliable end-to-end pipelines. We propose DC-Check, an actionable checklist-style framework to elicit data-centric considerations. This data-centric lens on development aims to promote thoughtfulness and transparency prior to system development.
  arXiv  Detail & Related papers  (2022-11-09T17:32:09Z)
• Paradigm selection for Data Fusion of SAR and Multispectral Sentinel data applied to Land-Cover Classification [63.072664304695465]
  In this letter, four data fusion paradigms, based on Convolutional Neural Networks (CNNs) are analyzed and implemented. The goals are to provide a systematic procedure for choosing the best data fusion framework, resulting in the best classification results. The procedure has been validated for land-cover classification but it can be transferred to other cases.
  arXiv  Detail & Related papers  (2021-06-18T11:36:54Z)
• Deep Learning based Pedestrian Inertial Navigation: Methods, Dataset and On-Device Inference [49.88536971774444]
  Inertial measurements units (IMUs) are small, cheap, energy efficient, and widely employed in smart devices and mobile robots. Exploiting inertial data for accurate and reliable pedestrian navigation supports is a key component for emerging Internet-of-Things applications and services. We present and release the Oxford Inertial Odometry dataset (OxIOD), a first-of-its-kind public dataset for deep learning based inertial navigation research.
  arXiv  Detail & Related papers  (2020-01-13T04:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.