Multi-view Contrastive Coding of Remote Sensing Images at Pixel-level

• URL: http://arxiv.org/abs/2105.08501v1
• Date: Tue, 18 May 2021 13:28:46 GMT
• Title: Multi-view Contrastive Coding of Remote Sensing Images at Pixel-level
• Authors: Yuxing Chen
• Abstract summary: A pixel-wise contrastive approach based on an unlabeled multi-view setting is proposed to overcome this limitation. A pseudo-Siamese ResUnet is trained to learn a representation that aims to align features from the shifted positive pairs. Results demonstrate both improvements in efficiency and accuracy over the state-of-the-art multi-view contrastive methods.
• Score: 5.64497799927668
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Our planet is viewed by satellites through multiple sensors (e.g., multi-spectral, Lidar and SAR) and at different times. Multi-view observations bring us complementary information than the single one. Alternatively, there are common features shared between different views, such as geometry and semantics. Recently, contrastive learning methods have been proposed for the alignment of multi-view remote sensing images and improving the feature representation of single sensor images by modeling view-invariant factors. However, these methods are based on the pretraining of the predefined tasks or just focus on image-level classification. Moreover, these methods lack research on uncertainty estimation. In this work, a pixel-wise contrastive approach based on an unlabeled multi-view setting is proposed to overcome this limitation. This is achieved by the use of contrastive loss in the feature alignment and uniformity between multi-view images. In this approach, a pseudo-Siamese ResUnet is trained to learn a representation that aims to align features from the shifted positive pairs and uniform the induced distribution of the features on the hypersphere. The learned features of multi-view remote sensing images are evaluated on a liner protocol evaluation and an unsupervised change detection task. We analyze key properties of the approach that make it work, finding that the requirement of shift equivariance ensured the success of the proposed approach and the uncertainty estimation of representations leads to performance improvements. Moreover, the performance of multi-view contrastive learning is affected by the choice of different sensors. Results demonstrate both improvements in efficiency and accuracy over the state-of-the-art multi-view contrastive methods.

Related papers

• MODEL&CO: Exoplanet detection in angular differential imaging by learning across multiple observations [37.845442465099396]
  Most post-processing methods build a model of the nuisances from the target observations themselves. We propose to build the nuisance model from an archive of multiple observations by leveraging supervised deep learning techniques. We apply the proposed algorithm to several datasets from the VLT/SPHERE instrument, and demonstrate a superior precision-recall trade-off.
  arXiv  Detail & Related papers  (2024-09-23T09:22:45Z)
• ProbMCL: Simple Probabilistic Contrastive Learning for Multi-label Visual Classification [16.415582577355536]
  Multi-label image classification presents a challenging task in many domains, including computer vision and medical imaging. Recent advancements have introduced graph-based and transformer-based methods to improve performance and capture label dependencies. We propose Probabilistic Multi-label Contrastive Learning (ProbMCL), a novel framework to address these challenges.
  arXiv  Detail & Related papers  (2024-01-02T22:15:20Z)
• Reinforcement-based Display-size Selection for Frugal Satellite Image Change Detection [5.656581242851759]
  We introduce a novel interactive satellite image change detection algorithm based on active learning. The proposed method is iterative and consists in frugally probing the user (oracle) about the labels of the most critical images.
  arXiv  Detail & Related papers  (2023-12-28T11:14:43Z)
• Two Approaches to Supervised Image Segmentation [55.616364225463066]
  The present work develops comparison experiments between deep learning and multiset neurons approaches. The deep learning approach confirmed its potential for performing image segmentation. The alternative multiset methodology allowed for enhanced accuracy while requiring little computational resources.
  arXiv  Detail & Related papers  (2023-07-19T16:42:52Z)
• Learning from Multi-Perception Features for Real-Word Image Super-resolution [87.71135803794519]
  We propose a novel SR method called MPF-Net that leverages multiple perceptual features of input images. Our method incorporates a Multi-Perception Feature Extraction (MPFE) module to extract diverse perceptual information. We also introduce a contrastive regularization term (CR) that improves the model's learning capability.
  arXiv  Detail & Related papers  (2023-05-26T07:35:49Z)
• Unsupervised Feature Clustering Improves Contrastive Representation Learning for Medical Image Segmentation [18.75543045234889]
  Self-supervised instance discrimination is an effective contrastive pretext task to learn feature representations and address limited medical image annotations. We propose a new self-supervised contrastive learning method that uses unsupervised feature clustering to better select positive and negative image samples. Our method outperforms state-of-the-art self-supervised contrastive techniques on these tasks.
  arXiv  Detail & Related papers  (2022-11-15T22:54:29Z)
• Modulated Contrast for Versatile Image Synthesis [60.304183493234376]
  MoNCE is a versatile metric that introduces image contrast to learn a calibrated metric for the perception of multifaceted inter-image distances. We introduce optimal transport in MoNCE to modulate the pushing force of negative samples collaboratively across multiple contrastive objectives.
  arXiv  Detail & Related papers  (2022-03-17T14:03:46Z)
• Learning Contrastive Representation for Semantic Correspondence [150.29135856909477]
  We propose a multi-level contrastive learning approach for semantic matching. We show that image-level contrastive learning is a key component to encourage the convolutional features to find correspondence between similar objects.
  arXiv  Detail & Related papers  (2021-09-22T18:34:14Z)
• Learning Discriminative Representations for Multi-Label Image Recognition [13.13795708478267]
  We propose a unified deep network to learn discriminative features for the multi-label task. By regularizing the whole network with the proposed loss, the performance of applying the wellknown ResNet-101 is improved significantly.
  arXiv  Detail & Related papers  (2021-07-23T12:10:46Z)
• Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
  Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries. This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management. We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
  arXiv  Detail & Related papers  (2020-10-12T13:26:30Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.