A data-flow oriented software architecture for heterogeneous marine data streams

• URL: http://arxiv.org/abs/2407.13231v1
• Date: Thu, 18 Jul 2024 07:40:19 GMT
• Title: A data-flow oriented software architecture for heterogeneous marine data streams
• Abstract summary: Various public organizations have collected, managed, and openly shared in-situ marine data in the past decade. There is no clear understanding of the impact of data quality in the engineering of systems. We propose main architectural decisions and a data flow-oriented component and connector view for marine in-situ data streams.
• Score: 9.327454646525855
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Marine in-situ data is collected by sensors mounted on fixed or mobile systems deployed into the ocean. This type of data is crucial both for the ocean industries and public authorities, e.g., for monitoring and forecasting the state of marine ecosystems and/or climate changes. Various public organizations have collected, managed, and openly shared in-situ marine data in the past decade. Recently, initiatives like the Ocean Decade Corporate Data Group have incentivized the sharing of marine data of public interest from private companies aiding in ocean management. However, there is no clear understanding of the impact of data quality in the engineering of systems, as well as on how to manage and exploit the collected data. In this paper, we propose main architectural decisions and a data flow-oriented component and connector view for marine in-situ data streams. Our results are based on a longitudinal empirical software engineering process, and driven by knowledge extracted from the experts in the marine domain from public and private organizations, and challenges identified in the literature. The proposed software architecture is instantiated and exemplified in a prototype implementation.

Related papers

• Seeing above the waves: A modular sensing framework for data acquisition at sea [0.35180162330725556]
  This work addresses the question: How can we design a modular and reproducible sensor platform for maritime autonomy?<n>We present a comprehensive design blueprint that incorporates diverse modalities - RADAR, LiDAR, IMU, AIS, RGB and LWIR cameras, and weather sensors.<n>Our modular platform enables long-term data collection, hardware-in-the-loop testing, and integration with existing sensors and algorithms.
  arXiv  Detail & Related papers  (2026-08-11T14:48:34Z)
• Compass: Navigating Global Marine Lead Data Integration through Expert-Guided LLM Agent [55.77954024801206]
  Marine lead (Pb) and its isotopes are critical tracers for ocean circulation and anthropogenic pollution.<n>Manual extraction is unscalable, while general-purpose Large Language Models (LLMs) lack the necessary domain-specific knowledge.<n>We introduce an expert-guided adaptation approach that enables LLMs to perform rigorous scientific data extraction without fine-tuning.
  arXiv  Detail & Related papers  (2026-05-28T14:06:25Z)
• OceanPile: A Large-Scale Multimodal Ocean Corpus for Foundation Models [57.19693589926157]
  OceanPile is a large-scale multimodal corpus designed for ocean foundation models.<n>It comprises three key components: OceanCorpus, OceanInstruction, and OceanBenchmark.<n>All datasets are publicly released to advance the field of marine artificial intelligence.
  arXiv  Detail & Related papers  (2026-04-25T14:53:37Z)
• OceanMAE: A Foundation Model for Ocean Remote Sensing [7.641383692351135]
  Ocean remote sensing is essential for applications such as bathymetry estimation, seabed characterization, marine litter detection, and ecosystem monitoring.<n>We propose OceanMAE, an ocean-specific masked autoencoder that extends standard MAE pre-training by integrating multispectral Sentinel-2 observations with physically meaningful ocean descriptors during self-supervised learning.<n>Experiments show that OceanMAE yields the strongest gains on marine segmentation, while bathymetry benefits are competitive and task-dependent.
  arXiv  Detail & Related papers  (2026-04-09T12:29:41Z)
• OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation [55.978228064498865]
  We present OceanSAR-2, the second generation of our foundation model for SAR-based ocean observation.<n>Building on our earlier release, which pioneered self-supervised learning on Sentinel-1 Wave Mode data, OceanSAR-2 relies on improved SSL training and dynamic data curation strategies.
  arXiv  Detail & Related papers  (2026-01-12T10:20:43Z)
• Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection [54.1960918379255]
  Neptune-X is a data-centric generative-selection framework for maritime object detection.<n>X-to-Maritime is a multi-modality-conditioned generative model that synthesizes diverse and realistic maritime scenes.<n>Our approach sets a new benchmark in maritime scene synthesis, significantly improving detection accuracy.
  arXiv  Detail & Related papers  (2025-09-25T04:59:02Z)
• OKG-LLM: Aligning Ocean Knowledge Graph with Observation Data via LLMs for Global Sea Surface Temperature Prediction [70.48962924608033]
  This work presents the first systematic effort to construct an Ocean Knowledge Graph (OKG) specifically designed to represent diverse ocean knowledge for SST prediction.<n>We develop a graph embedding network to learn the comprehensive semantic and structural knowledge within the OKG, capturing both the unique characteristics of individual sea regions and the complex correlations between them. Finally, we align the learned knowledge with fine-grained numerical SST data and leverage a pre-trained LLM to model SST patterns for accurate prediction.
  arXiv  Detail & Related papers  (2025-07-31T02:06:03Z)
• MLOps with Microservices: A Case Study on the Maritime Domain [0.10923877073891444]
  This case study describes challenges and lessons learned on building Ocean Guard: a Machine Learning-Enabled System (MLES) for anomaly detection in the maritime domain.<n>As a MLES, Ocean Guard employs code, model, and data contracts to establish guidelines between its services.<n>This case study hopes to inspire software engineers, machine learning engineers, and data scientists to leverage similar approaches for their systems.
  arXiv  Detail & Related papers  (2025-06-06T16:04:59Z)
• Introducing VaDA: Novel Image Segmentation Model for Maritime Object Segmentation Using New Dataset [3.468621550644668]
  The maritime shipping industry is undergoing rapid evolution driven by advancements in computer vision artificial intelligence (AI) object recognition in maritime environments faces challenges such as light reflection, interference, intense lighting, and various weather conditions. Existing AI recognition models and datasets have limited suitability for composing autonomous navigation systems.
  arXiv  Detail & Related papers  (2024-07-12T05:48:53Z)
• Multi-modal Perception Dataset of In-water Objects for Autonomous Surface Vehicles [10.732732686425308]
  This paper introduces the first publicly accessible multi-modal perception dataset for autonomous maritime navigation. It focuses on in-water obstacles within the aquatic environment to enhance situational awareness for Autonomous Surface Vehicles (ASVs)
  arXiv  Detail & Related papers  (2024-04-29T04:00:19Z)
• A2CI: A Cloud-based, Service-oriented Geospatial Cyberinfrastructure to Support Atmospheric Research [5.706655778525718]
  This paper reports the outcomes of an NSF-funded project that developed a geospatial cyberinfrastructure to support atmospheric research. We first introduce the service-oriented system framework then describe in detail the implementation of the data discovery module, data management module, data integration module, data analysis and visualization modules following the cloud computing principles.
  arXiv  Detail & Related papers  (2024-03-15T08:28:38Z)
• Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future [130.87142103774752]
  This review systematically assesses over seventy open-source autonomous driving datasets. It offers insights into various aspects, such as the principles underlying the creation of high-quality datasets. It also delves into the scientific and technical challenges that warrant resolution.
  arXiv  Detail & Related papers  (2023-12-06T10:46:53Z)
• OceanBench: The Sea Surface Height Edition [5.307677318971956]
  Ocean satellite data presents challenges for information extraction due to their sparsity and irregular sampling, signal complexity, and noise. Machine learning (ML) techniques have demonstrated their capabilities in dealing with large-scale, complex signals. OceanBench is a unifying framework that provides standardized processing steps that comply with domain-expert standards.
  arXiv  Detail & Related papers  (2023-09-27T12:00:40Z)
• Toward the Automated Construction of Probabilistic Knowledge Graphs for the Maritime Domain [60.76554773885988]
  International maritime crime is becoming increasingly sophisticated, often associated with wider criminal networks. This has led to research and development efforts aimed at combining hard data with other types of data. We propose Maritime DeepDive, an initial prototype for the automated construction of probabilistic knowledge graphs.
  arXiv  Detail & Related papers  (2023-05-04T00:24:30Z)
• Multimodal Dataset from Harsh Sub-Terranean Environment with Aerosol Particles for Frontier Exploration [55.41644538483948]
  This paper introduces a multimodal dataset from the harsh and unstructured underground environment with aerosol particles. It contains synchronized raw data measurements from all onboard sensors in Robot Operating System (ROS) format. The focus of this paper is not only to capture both temporal and spatial data diversities but also to present the impact of harsh conditions on captured data.
  arXiv  Detail & Related papers  (2023-04-27T20:21:18Z)
• Towards Avoiding the Data Mess: Industry Insights from Data Mesh Implementations [1.5029560229270191]
  Data mesh is a socio-technical, decentralized, distributed concept for enterprise data management. We conduct 15 semi-structured interviews with industry experts. Our findings synthesize insights from industry experts and provide researchers and professionals with preliminary guidelines for the successful adoption of data mesh.
  arXiv  Detail & Related papers  (2023-02-03T13:09:57Z)
• Learning-based estimation of in-situ wind speed from underwater acoustics [58.293528982012255]
  We introduce a deep learning approach for the retrieval of wind speed time series from underwater acoustics. Our approach bridges data assimilation and learning-based frameworks to benefit both from prior physical knowledge and computational efficiency.
  arXiv  Detail & Related papers  (2022-08-18T15:27:40Z)
• Urban Sensing based on Mobile Phone Data: Approaches, Applications and Challenges [67.71975391801257]
  Much concern in mobile data analysis is related to human beings and their behaviours. This work aims to review the methods and techniques that have been implemented to discover knowledge from mobile phone data.
  arXiv  Detail & Related papers  (2020-08-29T15:14:03Z)
• A Data Scientist's Guide to Streamflow Prediction [55.22219308265945]
  We focus on the element of hydrologic rainfall--runoff models and their application to forecast floods and predict streamflow. This guide aims to help interested data scientists gain an understanding of the problem, the hydrologic concepts involved, and the details that come up along the way.
  arXiv  Detail & Related papers  (2020-06-05T08:04:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.