Multitemporal and multispectral data fusion for super-resolution of Sentinel-2 images

• URL: http://arxiv.org/abs/2301.11154v1
• Date: Thu, 26 Jan 2023 15:01:25 GMT
• Title: Multitemporal and multispectral data fusion for super-resolution of Sentinel-2 images
• Authors: Tomasz Tarasiewicz, Jakub Nalepa, Reuben A. Farrugia, Gianluca Valentino, Mang Chen, Johann A. Briffa, Michal Kawulok
• Abstract summary: DeepSent is a new deep network for super-resolving multitemporal series of Sentinel-2 images. We show that our solution outperforms other state-of-the-art techniques that realize either multitemporal or multispectral data fusion. We have applied our method to super-resolve real-world Sentinel-2 images, enhancing the spatial resolution of all the spectral bands to 3.3 m nominal ground sampling distance.
• Score: 11.169492436455423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multispectral Sentinel-2 images are a valuable source of Earth observation data, however spatial resolution of their spectral bands limited to 10 m, 20 m, and 60 m ground sampling distance remains insufficient in many cases. This problem can be addressed with super-resolution, aimed at reconstructing a high-resolution image from a low-resolution observation. For Sentinel-2, spectral information fusion allows for enhancing the 20 m and 60 m bands to the 10 m resolution. Also, there were attempts to combine multitemporal stacks of individual Sentinel-2 bands, however these two approaches have not been combined so far. In this paper, we introduce DeepSent -- a new deep network for super-resolving multitemporal series of multispectral Sentinel-2 images. It is underpinned with information fusion performed simultaneously in the spectral and temporal dimensions to generate an enlarged multispectral image. In our extensive experimental study, we demonstrate that our solution outperforms other state-of-the-art techniques that realize either multitemporal or multispectral data fusion. Furthermore, we show that the advantage of DeepSent results from how these two fusion types are combined in a single architecture, which is superior to performing such fusion in a sequential manner. Importantly, we have applied our method to super-resolve real-world Sentinel-2 images, enhancing the spatial resolution of all the spectral bands to 3.3 m nominal ground sampling distance, and we compare the outcome with very high-resolution WorldView-2 images. We will publish our implementation upon paper acceptance, and we expect it will increase the possibilities of exploiting super-resolved Sentinel-2 images in real-life applications.

Related papers

• Cross-sensor super-resolution of irregularly sampled Sentinel-2 time series [2.9748898344267785]
  We introduce BreizhSR, a new dataset for 4x super-resolution of Sentinel-2 time series. We show that using multiple images significantly improves super-resolution performance. We observe a trade-off between spectral fidelity and perceptual quality of the reconstructed HR images.
  arXiv  Detail & Related papers  (2024-04-25T08:36:09Z)
• Object Detection in Hyperspectral Image via Unified Spectral-Spatial Feature Aggregation [55.9217962930169]
  We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images. S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
  arXiv  Detail & Related papers  (2023-06-14T09:01:50Z)
• Passive superresolution imaging of incoherent objects [63.942632088208505]
  Method consists of measuring the field's spatial mode components in the image plane in the overcomplete basis of Hermite-Gaussian modes and their superpositions. Deep neural network is used to reconstruct the object from these measurements.
  arXiv  Detail & Related papers  (2023-04-19T15:53:09Z)
• MuS2: A Benchmark for Sentinel-2 Multi-Image Super-Resolution [6.480645418615952]
  Insufficient spatial resolution of satellite imagery, including Sentinel-2 data, is a serious limitation in many practical use cases. Super-resolution reconstruction is receiving considerable attention from the remote sensing community. We introduce a new MuS2 benchmark for multi-image super-resolution reconstruction of Sentinel-2 images.
  arXiv  Detail & Related papers  (2022-10-06T08:29:54Z)
• PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening [50.943080184828524]
  We propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously.
  arXiv  Detail & Related papers  (2022-07-29T03:09:21Z)
• Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusion [67.35540259040806]
  We propose a subpixel-level HS super-resolution framework by devising a novel decoupled-and-coupled network, called DCNet. As the name suggests, DC-Net first decouples the input into common (or cross-sensor) and sensor-specific components. We append a self-supervised learning module behind the CSU net by guaranteeing the material consistency to enhance the detailed appearances of the restored HS product.
  arXiv  Detail & Related papers  (2022-05-07T23:40:36Z)
• Dual-Stage Approach Toward Hyperspectral Image Super-Resolution [21.68598210467761]
  We propose a new structure for hyperspectral image super-resolution (DualSR) In coarse stage, five bands with high similarity in a certain spectral range are divided into three groups, and the current band is guided to study the potential knowledge. In fine stage, an enhanced back-projection method via spectral angle constraint is developed to learn the content of spatial-spectral consistency.
  arXiv  Detail & Related papers  (2022-04-09T04:36:44Z)
• Multi-Spectral Multi-Image Super-Resolution of Sentinel-2 with Radiometric Consistency Losses and Its Effect on Building Delineation [23.025397327720874]
  We present the first results of applying multi-image super-resolution (MISR) to multi-spectral remote sensing imagery. We show that MISR is superior to single-image super-resolution and other baselines on a range of image fidelity metrics.
  arXiv  Detail & Related papers  (2021-11-05T02:49:04Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)
• HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery [55.253395881190436]
  Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem. This is important for satellite monitoring of human impact on the planet. We present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion.
  arXiv  Detail & Related papers  (2020-02-15T22:17:47Z)
• Hyperspectral and multispectral image fusion under spectrally varying spatial blurs -- Application to high dimensional infrared astronomical imaging [11.243400478302767]
  We propose a data fusion method which combines the benefits of each image to recover a high-spectral resolution data variant. We conduct experiments on a realistic synthetic dataset of simulated observation of the upcoming James Webb Space Telescope.
  arXiv  Detail & Related papers  (2019-12-26T13:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.