SkySeg: Collaborative Onboard Semantic Segmentation with Heterogeneous UAVs in the Wild

• URL: http://arxiv.org/abs/2605.24014v1
• Date: Wed, 20 May 2026 03:10:52 GMT
• Title: SkySeg: Collaborative Onboard Semantic Segmentation with Heterogeneous UAVs in the Wild
• Abstract summary: This paper introduces SkySeg, a heterogeneous multi-UAV air-air cooperation framework to enable onboard semantic segmentation using low-cost sensors.<n> Experimental results demonstrate that our SkySeg framework accelerates inference latency by approximately 3.6x, improves onboard segmentation accuracy by 5.91%, and achieves a 10.91% average accuracy gain in the wild.
• Score: 13.78779735589176
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The demand for unmanned aerial vehicle (UAV)-based image acquisition and analysis has surged, with UAVs increasingly utilized for semantic segmentation tasks. To meet the real-time analysis requirements of UAV remote sensing missions, performing onboard computation and making decisions based on the results is a natural approach. However, deploying semantic segmentation on resource-constrained UAV platforms presents two significant challenges: 1) hardware constraints limit the ability of UAVs to perform real-time semantic segmentation, and 2) environmental variations during flight cause data distribution shifts, deviating from the original training data. To address these issues, this paper introduces SkySeg, a heterogeneous multi-UAV air-air cooperation framework that integrates computer vision and flight pattern to enable onboard semantic segmentation using low-cost sensors. SkySeg employs an efficient information fusion inference method, combining low-definition, wide-area images with high-definition, focused-area images. Additionally, it incorporates a cross-device test-time adaptation (TTA) strategy to enhance segmentation performance in dynamic environments by collaboratively addressing distribution shifts of test data streams across UAVs. Experimental results demonstrate that our SkySeg framework accelerates inference latency by approximately 3.6x, improves onboard segmentation accuracy by 5.91\%, and achieves a 10.91\% average accuracy gain in the wild.

Related papers

• DESCENT: Directed Edge Scene Encoding for Airport Surface Movement Prediction [54.94692733670454]
  DESCENT is a transformer-based architecture designed to handle heterogeneous dynamics and strict topological constraints.<n>Our approach features a Potential Reachable Set (PRS) context sampling mechanism that adaptively collects airfield environment context.<n>Our evaluations on the Amelia-10 benchmark demonstrate significant performance improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-08-26T16:49:50Z)
• DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images [50.96240625673857]
  DroneFINE is a novel PEFT paradigm comprising two domain-aware complementary modules tailored for VLM-based drone image detectors.<n>DroneFINE significantly outperforms existing PEFT methods and achieves performance comparable to full fine-tuning.
  arXiv  Detail & Related papers  (2026-07-01T02:27:40Z)
• SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion [76.8430169655641]
  spatial intelligence is essential for low-altitude unmanned aerial vehicle (UAV) perception, collaboration, and navigation.<n>We introduce SpatialUAV, a real low-altitude UAV benchmark covering semantic discrimination, spatial relation, aerial-aerial collaboration, aerial-ground collaboration, and motion understanding.<n>We show that current models remain far from human-level performance, with pronounced bottlenecks in cross-view association, structured grounding, geometric reasoning, and temporal viewpoint understanding.
  arXiv  Detail & Related papers  (2026-06-26T09:16:55Z)
• FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales [65.4821703903285]
  FLORO is a multimodal foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus.<n> FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and UAV-derived data.<n>We evaluated FLORO on the PANGAEA benchmark under a frozen-encoder protocol across scene classification, segmentation, and regression tasks.
  arXiv  Detail & Related papers  (2026-05-27T08:55:54Z)
• Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks [51.323657802729535]
  Multi-uncrewed aerial vehicles (UAV) cooperative perception has emerged as a promising paradigm for diverse low-altitude economy applications.<n>This paper proposes a communication-efficient cooperative perception framework, termed Base-Station-Helped UAV (BHU)<n>We employ a Top-K selection mechanism to identify the most informative pixels from UAV-captured RGB images, enabling sparsified visual transmission with reduced data volume and latency.
  arXiv  Detail & Related papers  (2026-03-13T17:49:28Z)
• STVG-R1: Incentivizing Instance-Level Reasoning and Grounding in Videos via Reinforcement Learning [65.36458157092207]
  In vision-language models (VLMs), misalignment between textual descriptions and visual coordinates often induces hallucinations.<n>We propose a novel visual prompting paradigm that avoids the difficult problem of aligning coordinates across modalities.<n>We introduce STVG-R1, the first reinforcement learning framework for STVG, which employs a task-driven reward to jointly optimize temporal accuracy, spatial consistency, and structural format regularization.
  arXiv  Detail & Related papers  (2026-02-12T08:53:32Z)
• AdaptFly: Prompt-Guided Adaptation of Foundation Models for Low-Altitude UAV Networks [10.80018338292861]
  Low-altitude Unmanned Aerial Vehicle (UAV) networks rely on robust semantic segmentation as a foundational enabler for distributed sensing-communication-control co-design.<n>We propose AdaptFly, a prompt-guided test-time adaptation framework that adjusts segmentation models without weight updates.<n>Experiments on UAVid and VDD benchmarks, along with real-world UAV deployments under diverse weather conditions, demonstrate that AdaptFly significantly improves segmentation accuracy and robustness.
  arXiv  Detail & Related papers  (2025-11-13T00:20:37Z)
• Trajectory Design for UAV-Based Low-Altitude Wireless Networks in Unknown Environments: A Digital Twin-Assisted TD3 Approach [62.11847362756054]
  Unmanned aerial vehicles (UAVs) are emerging as key enablers for low-altitude wireless network (LAWN)<n>We propose a digital twin (DT)-assisted training and deployment framework.<n>In this framework, the UAV transmits integrated sensing and communication signals to provide communication services to ground users, while simultaneously collecting echoes that are uploaded to the DT server to progressively construct virtual environments (VEs)<n>These VEs accelerate model training and are continuously updated with real-time UAV sensing data during deployment, supporting decision-making and enhancing flight safety.
  arXiv  Detail & Related papers  (2025-10-28T10:05:53Z)
• MobileGeo: Exploring Hierarchical Knowledge Distillation for Resource-Efficient Cross-view Drone Geo-Localization [47.16612614191333]
  Cross-view geo-localization enables drone localization by matching aerial images to geo-tagged satellite databases.<n>MobileGeo is a mobile-friendly framework designed for efficient on-device CVGL.<n>MobileGeo runs at 251.5 FPS on an NVIDIA AGX Orin edge device, demonstrating its practical viability for real-time on-device drone geo-localization.
  arXiv  Detail & Related papers  (2025-10-26T08:47:20Z)
• A Transformer-Based Conditional GAN with Multiple Instance Learning for UAV Signal Detection and Classification [17.586093539522327]
  This paper proposes a novel framework that integrates a Transformer-based Generative Adversarial Network (GAN) with Multiple Instance Locally Explainable Learning (MILET)<n> Experimental results show that the proposed method achieves superior accuracy 96.5% on the DroneDetect dataset and 98.6% on the DroneRF dataset.<n>The framework also demonstrates strong computational efficiency and robust generalization across diverse UAV platforms and flight states.
  arXiv  Detail & Related papers  (2025-07-19T12:35:45Z)
• Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmark [15.405137983083875]
  Aerial-ground cooperation offers a promising solution by integrating UAVs' aerial views with ground vehicles' local observations.<n>This paper presents a comprehensive solution for aerial-ground cooperative 3D perception through three key contributions.
  arXiv  Detail & Related papers  (2025-03-10T07:00:07Z)
• Semantic Segmentation of Unmanned Aerial Vehicle Remote Sensing Images using SegFormer [0.14999444543328289]
  This paper evaluates the effectiveness and efficiency of SegFormer, a semantic segmentation framework, for the semantic segmentation of UAV images. SegFormer variants, ranging in real-time (B0) to high-performance (B5) models, are assessed using the UAVid dataset tailored for semantic segmentation tasks. Experimental results showcase the model's performance on benchmark dataset, highlighting its ability to accurately delineate objects and land cover features in diverse UAV scenarios.
  arXiv  Detail & Related papers  (2024-10-01T21:40:15Z)
• Diffusion-Based Particle-DETR for BEV Perception [94.88305708174796]
  Bird-Eye-View (BEV) is one of the most widely-used scene representations for visual perception in Autonomous Vehicles (AVs) Recent diffusion-based methods offer a promising approach to uncertainty modeling for visual perception but fail to effectively detect small objects in the large coverage of the BEV. Here, we address this problem by combining the diffusion paradigm with current state-of-the-art 3D object detectors in BEV.
  arXiv  Detail & Related papers  (2023-12-18T09:52:14Z)
• Integrated Sensing, Computation, and Communication for UAV-assisted Federated Edge Learning [52.7230652428711]
  Federated edge learning (FEEL) enables privacy-preserving model training through periodic communication between edge devices and the server. Unmanned Aerial Vehicle (UAV)mounted edge devices are particularly advantageous for FEEL due to their flexibility and mobility in efficient data collection.
  arXiv  Detail & Related papers  (2023-06-05T16:01:33Z)
• Adaptive Path Planning for UAVs for Multi-Resolution Semantic Segmentation [28.104584236205405]
  A key challenge is planning missions to maximize the value of acquired data in large environments. This is, for example, relevant for monitoring agricultural fields. We propose an online planning algorithm which adapts the UAV paths to obtain high-resolution semantic segmentations.
  arXiv  Detail & Related papers  (2022-03-03T11:03:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.