Rapid Distributed Fine-tuning of a Segmentation Model Onboard Satellites

• URL: http://arxiv.org/abs/2411.17831v1
• Date: Tue, 26 Nov 2024 19:11:36 GMT
• Title: Rapid Distributed Fine-tuning of a Segmentation Model Onboard Satellites
• Authors: Meghan Plumridge, Rasmus Maråk, Chiara Ceccobello, Pablo Gómez, Gabriele Meoni, Filip Svoboda, Nicholas D. Lane,
• Abstract summary: This study presents a proof-of-concept using MobileSAM, a lightweight, pre-trained segmentation model, onboard Unibap iX10-100 satellite hardware. Our research investigates the potential of fine-tuning MobileSAM in a decentralised way onboard multiple satellites in rapid response to a disaster.
• Score: 13.235981880457125
• License:
• Abstract: Segmentation of Earth observation (EO) satellite data is critical for natural hazard analysis and disaster response. However, processing EO data at ground stations introduces delays due to data transmission bottlenecks and communication windows. Using segmentation models capable of near-real-time data analysis onboard satellites can therefore improve response times. This study presents a proof-of-concept using MobileSAM, a lightweight, pre-trained segmentation model, onboard Unibap iX10-100 satellite hardware. We demonstrate the segmentation of water bodies from Sentinel-2 satellite imagery and integrate MobileSAM with PASEOS, an open-source Python module that simulates satellite operations. This integration allows us to evaluate MobileSAM's performance under simulated conditions of a satellite constellation. Our research investigates the potential of fine-tuning MobileSAM in a decentralised way onboard multiple satellites in rapid response to a disaster. Our findings show that MobileSAM can be rapidly fine-tuned and benefits from decentralised learning, considering the constraints imposed by the simulated orbital environment. We observe improvements in segmentation performance with minimal training data and fast fine-tuning when satellites frequently communicate model updates. This study contributes to the field of onboard AI by emphasising the benefits of decentralised learning and fine-tuning pre-trained models for rapid response scenarios. Our work builds on recent related research at a critical time; as extreme weather events increase in frequency and magnitude, rapid response with onboard data analysis is essential.

Related papers

• Low-altitude Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning [72.72954660774002]
  Low Earth Orbit (LEO) satellites can be used to assist maritime wireless communications for data transmission across wide-ranging areas. Extensive coverage of LEO satellites, combined with openness of channels, can cause the communication process to suffer from security risks. This paper presents a low-altitude friendly-jamming LEO satellite-maritime communication system enabled by a unmanned aerial vehicle.
  arXiv  Detail & Related papers  (2025-01-26T10:13:51Z)
• Enhancing Maritime Situational Awareness through End-to-End Onboard Raw Data Analysis [4.441792803766689]
  This research presents a framework addressing the strict bandwidth, energy, and latency constraints of small satellites. It investigates the application of deep learning techniques for direct ship detection and classification from raw satellite imagery. By simplifying the onboard processing chain, our approach facilitates direct analyses without requiring computationally intensive steps such as calibration and ortho-rectification.
  arXiv  Detail & Related papers  (2024-11-05T18:38:42Z)
• Cooperative Federated Learning over Ground-to-Satellite Integrated Networks: Joint Local Computation and Data Offloading [33.44828515877944]
  We propose a ground-to-satellite cooperative federated learning (FL) methodology to facilitate machine learning service management over remote regions. Our methodology orchestrates satellite constellations to provide the following key functions during FL. We show that our methodology can significantly speed up the convergence of FL compared with terrestrial-only and other satellite baseline approaches.
  arXiv  Detail & Related papers  (2023-12-23T22:09:31Z)
• FedSN: A Federated Learning Framework over Heterogeneous LEO Satellite Networks [18.213174641216884]
  A large number of Low Earth Orbit (LEO) satellites have been launched and deployed successfully in space by commercial companies, such as SpaceX. Due to multimodal sensors equipped by the LEO satellites, they serve not only for communication but also for various machine learning applications, such as space modulation recognition, remote sensing image classification, etc. We propose FedSN as a general FL framework to tackle the above challenges, and fully explore data diversity on LEO satellites.
  arXiv  Detail & Related papers  (2023-11-02T14:47:06Z)
• Diffusion Models for Interferometric Satellite Aperture Radar [73.01013149014865]
  Probabilistic Diffusion Models (PDMs) have recently emerged as a very promising class of generative models. Here, we leverage PDMs to generate several radar-based satellite image datasets. We show that PDMs succeed in generating images with complex and realistic structures, but that sampling time remains an issue.
  arXiv  Detail & Related papers  (2023-08-31T16:26:17Z)
• Fast model inference and training on-board of Satellites [16.93335252280199]
  This study deploys a lightweight foundational model called RaVAEn on D-Orbit's ION SCV004 satellite. RaVAEn generates compressed latent vectors from small image tiles, enabling several downstream tasks.
  arXiv  Detail & Related papers  (2023-07-17T17:59:09Z)
• One-Shot Federated Learning for LEO Constellations that Reduces Convergence Time from Days to 90 Minutes [3.096615629099617]
  A Low Earth orbit (LEO) satellite constellation consists of a large number of small satellites traveling in space with high mobility. Federated Learning (FL) is a promising approach because it eliminates the need for transmitting raw data and hence is both bandwidth and privacy-friendly. We propose a novel one-shot FL approach for LEO satellites, called LEOShot, that needs only a single communication round to complete the entire learning process.
  arXiv  Detail & Related papers  (2023-05-21T01:57:56Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Embedding Earth: Self-supervised contrastive pre-training for dense land cover classification [61.44538721707377]
  We present Embedding Earth a self-supervised contrastive pre-training method for leveraging the large availability of satellite imagery. We observe significant improvements up to 25% absolute mIoU when pre-trained with our proposed method. We find that learnt features can generalize between disparate regions opening up the possibility of using the proposed pre-training scheme.
  arXiv  Detail & Related papers  (2022-03-11T16:14:14Z)
• Deep Learning Aided Routing for Space-Air-Ground Integrated Networks Relying on Real Satellite, Flight, and Shipping Data [79.96177511319713]
  Current maritime communications mainly rely on satellites having meager transmission resources, hence suffering from poorer performance than modern terrestrial wireless networks. With the growth of transcontinental air traffic, the promising concept of aeronautical ad hoc networking relying on commercial passenger airplanes is potentially capable of enhancing satellite-based maritime communications via air-to-ground and multi-hop air-to-air links. We propose space-air-ground integrated networks (SAGINs) for supporting ubiquitous maritime communications, where the low-earth-orbit satellite constellations, passenger airplanes, terrestrial base stations, ships, respectively, serve as the space-, air-,
  arXiv  Detail & Related papers  (2021-10-28T14:12:10Z)
• Integrating LEO Satellite and UAV Relaying via Reinforcement Learning for Non-Terrestrial Networks [51.05735925326235]
  A mega-constellation of low-earth orbit (LEO) satellites has the potential to enable long-range communication with low latency. We study the problem of forwarding packets between two faraway ground terminals, through an LEO satellite selected from an orbiting constellation. To maximize the end-to-end data rate, the satellite association and HAP location should be optimized. We tackle this problem using deep reinforcement learning (DRL) with a novel action dimension reduction technique.
  arXiv  Detail & Related papers  (2020-05-26T05:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.