A transformer boosted UNet for smoke segmentation in complex backgrounds in multispectral LandSat imagery

• URL: http://arxiv.org/abs/2406.13105v1
• Date: Tue, 18 Jun 2024 23:38:24 GMT
• Title: A transformer boosted UNet for smoke segmentation in complex backgrounds in multispectral LandSat imagery
• Authors: Jixue Liu, Jiuyong Li, Stefan Peters, Liang Zhao,
• Abstract summary: Smokes present challenges in detection due to variations in density, color, lighting, and backgrounds such as clouds, haze, and/or mist. This paper proposes a new segmentation model called VTrUNet which consists of a virtual band construction module to capture spectral patterns.
• Score: 17.098729939840716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many studies have been done to detect smokes from satellite imagery. However, these prior methods are not still effective in detecting various smokes in complex backgrounds. Smokes present challenges in detection due to variations in density, color, lighting, and backgrounds such as clouds, haze, and/or mist, as well as the contextual nature of thin smoke. This paper addresses these challenges by proposing a new segmentation model called VTrUNet which consists of a virtual band construction module to capture spectral patterns and a transformer boosted UNet to capture long range contextual features. The model takes imagery of six bands: red, green, blue, near infrared, and two shortwave infrared bands as input. To show the advantages of the proposed model, the paper presents extensive results for various possible model architectures improving UNet and draws interesting conclusions including that adding more modules to a model does not always lead to a better performance. The paper also compares the proposed model with very recently proposed and related models for smoke segmentation and shows that the proposed model performs the best and makes significant improvements on prediction performances

Related papers

• SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models [0.0]
  Foundation models refer to deep learning models pretrained on large unlabeled datasets through self-supervised algorithms. Various foundation models have been developed for remote sensing, such as those for multispectral, high-resolution, and hyperspectral images. This research proposes SatMamba, a new pretraining framework that combines masked autoencoders with State Space Model.
  arXiv  Detail & Related papers  (2025-02-01T14:07:21Z)
• Mantis Shrimp: Exploring Photometric Band Utilization in Computer Vision Networks for Photometric Redshift Estimation [0.30924355683504173]
  We present a model for photometric redshift estimation that fuses ultra-violet (GALEX), optical (PanSTARRS), and infrared (UnWISE) imagery. Mantis Shrimp estimates the conditional density estimate of redshift using cutout images. We study how the models learn to use information across bands, finding evidence that our models successfully incorporates information from all surveys.
  arXiv  Detail & Related papers  (2025-01-15T19:46:23Z)
• LapGSR: Laplacian Reconstructive Network for Guided Thermal Super-Resolution [1.747623282473278]
  Fusing multiple modalities to produce high-resolution images often requires dense models with millions of parameters and a heavy computational load. We propose LapGSR, a multimodal, lightweight, generative model incorporating Laplacian image pyramids for guided thermal super-resolution.
  arXiv  Detail & Related papers  (2024-11-12T12:23:19Z)
• Fine Tuning Text-to-Image Diffusion Models for Correcting Anomalous Images [0.0]
  This study proposes a method to mitigate such issues by fine-tuning the Stable Diffusion 3 model using the DreamBooth technique. Experimental results targeting the prompt "lying on the grass/street" demonstrate that the fine-tuned model shows improved performance in visual evaluation and metrics such as Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Frechet Inception Distance (FID)
  arXiv  Detail & Related papers  (2024-09-23T00:51:47Z)
• MultiDiff: Consistent Novel View Synthesis from a Single Image [60.04215655745264]
  MultiDiff is a novel approach for consistent novel view synthesis of scenes from a single RGB image. Our results demonstrate that MultiDiff outperforms state-of-the-art methods on the challenging, real-world datasets RealEstate10K and ScanNet.
  arXiv  Detail & Related papers  (2024-06-26T17:53:51Z)
• Diff-Mosaic: Augmenting Realistic Representations in Infrared Small Target Detection via Diffusion Prior [63.64088590653005]
  We propose Diff-Mosaic, a data augmentation method based on the diffusion model. We introduce an enhancement network called Pixel-Prior, which generates highly coordinated and realistic Mosaic images. In the second stage, we propose an image enhancement strategy named Diff-Prior. This strategy utilizes diffusion priors to model images in the real-world scene.
  arXiv  Detail & Related papers  (2024-06-02T06:23:05Z)
• RANRAC: Robust Neural Scene Representations via Random Ray Consensus [12.161889666145127]
  RANdom RAy Consensus (RANRAC) is an efficient approach to eliminate the effect of inconsistent data. We formulate a fuzzy adaption of the RANSAC paradigm, enabling its application to large scale models. Results indicate significant improvements compared to state-of-the-art robust methods for novel-view synthesis.
  arXiv  Detail & Related papers  (2023-12-15T13:33:09Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• Breaking Through the Haze: An Advanced Non-Homogeneous Dehazing Method based on Fast Fourier Convolution and ConvNeXt [14.917290578644424]
  Haze usually leads to deteriorated images with low contrast, color shift and structural distortion. We propose a novel two branch network that leverages 2D discrete wavelete transform (DWT), fast Fourier convolution (FFC) residual block and a pretrained ConvNeXt model. Our model is able to effectively explore global contextual information and produce images with better perceptual quality.
  arXiv  Detail & Related papers  (2023-05-08T02:59:02Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations [77.90883737693325]
  This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for rendering in metric space.
  arXiv  Detail & Related papers  (2022-09-02T17:44:50Z)
• Visual Saliency Transformer [127.33678448761599]
  We develop a novel unified model based on a pure transformer, Visual Saliency Transformer (VST), for both RGB and RGB-D salient object detection (SOD) It takes image patches as inputs and leverages the transformer to propagate global contexts among image patches. Experimental results show that our model outperforms existing state-of-the-art results on both RGB and RGB-D SOD benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-25T08:24:06Z)
• Broad-UNet: Multi-scale feature learning for nowcasting tasks [3.9318191265352196]
  We treat the nowcasting problem as an image-to-image translation problem using satellite imagery. We introduce Broad-UNet, a novel architecture based on the core UNet model, to efficiently address this problem. The proposed model is applied on two different nowcasting tasks, i.e. precipitation maps and cloud cover nowcasting.
  arXiv  Detail & Related papers  (2021-02-12T11:06:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.