Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery

• URL: http://arxiv.org/abs/2310.04721v1
• Date: Sat, 7 Oct 2023 07:44:59 GMT
• Title: Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery
• Authors: Qi Li, Jiaxin Cai, Yuanlong Yu, Jason Gu, Jia Pan, Wenxi Liu
• Abstract summary: This paper explores the intricate problem of achieving efficient and effective segmentation of ultra-high resolution UAV imagery. We propose a GPU memory-efficient and effective framework for local inference without accessing the context beyond local patches. We present an efficient memory-based interaction scheme to correct potential semantic bias of the underlying high-resolution information.
• Score: 35.96063342025938
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Amidst the swift advancements in photography and sensor technologies, high-definition cameras have become commonplace in the deployment of Unmanned Aerial Vehicles (UAVs) for diverse operational purposes. Within the domain of UAV imagery analysis, the segmentation of ultra-high resolution images emerges as a substantial and intricate challenge, especially when grappling with the constraints imposed by GPU memory-restricted computational devices. This paper delves into the intricate problem of achieving efficient and effective segmentation of ultra-high resolution UAV imagery, while operating under stringent GPU memory limitation. The strategy of existing approaches is to downscale the images to achieve computationally efficient segmentation. However, this strategy tends to overlook smaller, thinner, and curvilinear regions. To address this problem, we propose a GPU memory-efficient and effective framework for local inference without accessing the context beyond local patches. In particular, we introduce a novel spatial-guided high-resolution query module, which predicts pixel-wise segmentation results with high quality only by querying nearest latent embeddings with the guidance of high-resolution information. Additionally, we present an efficient memory-based interaction scheme to correct potential semantic bias of the underlying high-resolution information by associating cross-image contextual semantics. For evaluation of our approach, we perform comprehensive experiments over public benchmarks and achieve superior performance under both conditions of small and large GPU memory usage limitations. We will release the model and codes in the future.

Related papers

• HRDecoder: High-Resolution Decoder Network for Fundus Image Lesion Segmentation [12.606794661369959]
  We propose HRDecoder, a simple High-Resolution Decoder network for fundus lesion segmentation. It integrates a high-resolution representation learning module to capture fine-grained local features and a high-resolution fusion module to fuse multi-scale predictions. Our method effectively improves the overall segmentation accuracy of fundus lesions while consuming reasonable memory and computational overhead, and maintaining satisfying inference speed.
  arXiv  Detail & Related papers  (2024-11-06T15:13:31Z)
• SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images [50.742420049839474]
  'SaccadeDet' is an innovative architecture for gigapixel-level object detection, inspired by the human eye saccadic movement. Our approach, evaluated on the PANDA dataset, achieves an 8x speed increase over the state-of-the-art methods. It also demonstrates significant potential in gigapixel-level pathology analysis through its application to Whole Slide Imaging.
  arXiv  Detail & Related papers  (2024-07-25T11:22:54Z)
• Real-Time Image Segmentation via Hybrid Convolutional-Transformer Architecture Search [49.81353382211113]
  We address the challenge of integrating multi-head self-attention into high resolution representation CNNs efficiently. We develop a multi-target multi-branch supernet method, which fully utilizes the advantages of high-resolution features. We present a series of model via Hybrid Convolutional-Transformer Architecture Search (HyCTAS) method that searched for the best hybrid combination of light-weight convolution layers and memory-efficient self-attention layers.
  arXiv  Detail & Related papers  (2024-03-15T15:47:54Z)
• On the Effect of Image Resolution on Semantic Segmentation [27.115235051091663]
  We show that a model capable of directly producing high-resolution segmentations can match the performance of more complex systems. Our approach leverages a bottom-up information propagation technique across various scales. We have rigorously tested our method using leading-edge semantic segmentation datasets.
  arXiv  Detail & Related papers  (2024-02-08T04:21:30Z)
• Low-Resolution Self-Attention for Semantic Segmentation [96.81482872022237]
  We introduce the Low-Resolution Self-Attention (LRSA) mechanism to capture global context at a significantly reduced computational cost. Our approach involves computing self-attention in a fixed low-resolution space regardless of the input image's resolution. We demonstrate the effectiveness of our LRSA approach by building the LRFormer, a vision transformer with an encoder-decoder structure.
  arXiv  Detail & Related papers  (2023-10-08T06:10:09Z)
• A Robust Morphological Approach for Semantic Segmentation of Very High Resolution Images [2.2230089845369085]
  We develop a robust pipeline that seamlessly extends any existing semantic segmentation algorithm to high resolution images. Our method does not require the ground truth annotations of the high resolution images. We show that the semantic segmentation results obtained by our method beat the existing state-of-the-art algorithms on high-resolution images.
  arXiv  Detail & Related papers  (2022-08-02T05:25:35Z)
• High Quality Segmentation for Ultra High-resolution Images [72.97958314291648]
  We propose the Continuous Refinement Model for the ultra high-resolution segmentation refinement task. Our proposed method is fast and effective on image segmentation refinement.
  arXiv  Detail & Related papers  (2021-11-29T11:53:06Z)
• BoundarySqueeze: Image Segmentation as Boundary Squeezing [104.43159799559464]
  We propose a novel method for fine-grained high-quality image segmentation of both objects and scenes. Inspired by dilation and erosion from morphological image processing techniques, we treat the pixel level segmentation problems as squeezing object boundary. Our method yields large gains on COCO, Cityscapes, for both instance and semantic segmentation and outperforms previous state-of-the-art PointRend in both accuracy and speed under the same setting.
  arXiv  Detail & Related papers  (2021-05-25T04:58:51Z)
• A Novel Upsampling and Context Convolution for Image Semantic Segmentation [0.966840768820136]
  Recent methods for semantic segmentation often employ an encoder-decoder structure using deep convolutional neural networks. We propose a dense upsampling convolution method based on guided filtering to effectively preserve the spatial information of the image in the network. We report a new record of 82.86% and 81.62% of pixel accuracy on ADE20K and Pascal-Context benchmark datasets, respectively.
  arXiv  Detail & Related papers  (2021-03-20T06:16:42Z)
• Superpixel Segmentation Based on Spatially Constrained Subspace Clustering [57.76302397774641]
  We consider each representative region with independent semantic information as a subspace, and formulate superpixel segmentation as a subspace clustering problem. We show that a simple integration of superpixel segmentation with the conventional subspace clustering does not effectively work due to the spatial correlation of the pixels. We propose a novel convex locality-constrained subspace clustering model that is able to constrain the spatial adjacent pixels with similar attributes to be clustered into a superpixel.
  arXiv  Detail & Related papers  (2020-12-11T06:18:36Z)
• CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinement [0.0]
  State-of-the-art semantic segmentation methods were almost exclusively trained on images within a fixed resolution range. We propose a novel approach to address the high-resolution segmentation problem without using any high-resolution training data.
  arXiv  Detail & Related papers  (2020-05-06T01:38:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.