Semantics from Space: Satellite-Guided Thermal Semantic Segmentation Annotation for Aerial Field Robots

• URL: http://arxiv.org/abs/2403.14056v1
• Date: Thu, 21 Mar 2024 00:59:35 GMT
• Title: Semantics from Space: Satellite-Guided Thermal Semantic Segmentation Annotation for Aerial Field Robots
• Authors: Connor Lee, Saraswati Soedarmadji, Matthew Anderson, Anthony J. Clark, Soon-Jo Chung,
• Abstract summary: We present a new method to automatically generate semantic segmentation annotations for thermal imagery captured from an aerial vehicle. This new capability overcomes the challenge of developing thermal semantic perception algorithms for field robots. Our approach can produce highly-precise semantic segmentation labels using low-resolution satellite land cover data for little-to-no cost.
• Score: 8.265009823753982
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a new method to automatically generate semantic segmentation annotations for thermal imagery captured from an aerial vehicle by utilizing satellite-derived data products alongside onboard global positioning and attitude estimates. This new capability overcomes the challenge of developing thermal semantic perception algorithms for field robots due to the lack of annotated thermal field datasets and the time and costs of manual annotation, enabling precise and rapid annotation of thermal data from field collection efforts at a massively-parallelizable scale. By incorporating a thermal-conditioned refinement step with visual foundation models, our approach can produce highly-precise semantic segmentation labels using low-resolution satellite land cover data for little-to-no cost. It achieves 98.5% of the performance from using costly high-resolution options and demonstrates between 70-160% improvement over popular zero-shot semantic segmentation methods based on large vision-language models currently used for generating annotations for RGB imagery. Code will be available at: https://github.com/connorlee77/aerial-auto-segment.

Related papers

• SegDesicNet: Lightweight Semantic Segmentation in Remote Sensing with Geo-Coordinate Embeddings for Domain Adaptation [0.5461938536945723]
  We propose a novel unsupervised domain adaptation technique for remote sensing semantic segmentation. Our proposed SegDesicNet module regresses the GRID positional encoding of the geo coordinates projected over the unit sphere to obtain the domain loss. Our algorithm seeks to reduce the modeling disparity between artificial neural networks and human comprehension of the physical world.
  arXiv  Detail & Related papers  (2025-03-11T11:01:18Z)
• Semantic Neural Radiance Fields for Multi-Date Satellite Data [4.174845397893041]
  We propose a satellite specific Neural Radiance Fields (NeRF) model capable to obtain a three-dimensional semantic representation of the scene. The model derives the output from a set of multi-date satellite images with corresponding pixel-wise semantic labels. We enhance the color prediction by utilizing the semantic information to address temporal image inconsistencies.
  arXiv  Detail & Related papers  (2025-02-24T09:26:48Z)
• Hierarchical Features Matter: A Deep Exploration of GAN Priors for Improved Dataset Distillation [51.44054828384487]
  We propose a novel parameterization method dubbed Hierarchical Generative Latent Distillation (H-GLaD) This method systematically explores hierarchical layers within the generative adversarial networks (GANs) In addition, we introduce a novel class-relevant feature distance metric to alleviate the computational burden associated with synthetic dataset evaluation.
  arXiv  Detail & Related papers  (2024-06-09T09:15:54Z)
• Homography Guided Temporal Fusion for Road Line and Marking Segmentation [73.47092021519245]
  Road lines and markings are frequently occluded in the presence of moving vehicles, shadow, and glare. We propose a Homography Guided Fusion (HomoFusion) module to exploit temporally-adjacent video frames for complementary cues. We show that exploiting available camera intrinsic data and ground plane assumption for cross-frame correspondence can lead to a light-weight network with significantly improved performances in speed and accuracy.
  arXiv  Detail & Related papers  (2024-04-11T10:26:40Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• Learning Semantic Segmentation with Query Points Supervision on Aerial Images [57.09251327650334]
  We present a weakly supervised learning algorithm to train semantic segmentation algorithms. Our proposed approach performs accurate semantic segmentation and improves efficiency by significantly reducing the cost and time required for manual annotation.
  arXiv  Detail & Related papers  (2023-09-11T14:32:04Z)
• Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles [6.3267657083758735]
  We present a new method to adapt an RGB-trained water segmentation network to target-domain aerial thermal imagery using online self-supervision. This new thermal capability enables current autonomous aerial robots operating in near-shore environments to perform tasks such as visual navigation, bathymetry, and flow tracking at night.
  arXiv  Detail & Related papers  (2023-07-18T07:35:28Z)
• Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
  We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space. Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias. During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
  arXiv  Detail & Related papers  (2023-01-20T07:36:29Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• ARTSeg: Employing Attention for Thermal images Semantic Segmentation [6.060020806741279]
  We have designed an attention-based Recurrent Convolution Network (RCNN) encoder-decoder architecture named ARTSeg for thermal semantic segmentation. The efficacy of the proposed method is evaluated on the available public dataset, showing better performance with other state-of-the-art methods in mean intersection over union (IoU)
  arXiv  Detail & Related papers  (2021-11-30T10:17:28Z)
• Real-time Semantic Segmentation with Fast Attention [94.88466483540692]
  We propose a novel architecture for semantic segmentation of high-resolution images and videos in real-time. The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism. We show that results on multiple datasets demonstrate superior performance with better accuracy and speed compared to existing approaches.
  arXiv  Detail & Related papers  (2020-07-07T22:37:16Z)
• HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images [26.749261270690425]
  Real-world driving scenarios entail adverse environmental conditions such as nighttime illumination or glare. We propose a multimodal semantic segmentation model that can be applied during daytime and nighttime. Besides RGB images, we leverage thermal images, making our network significantly more robust.
  arXiv  Detail & Related papers  (2020-03-10T11:36:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.