Context Matters: Leveraging Spatiotemporal Metadata for Semi-Supervised Learning on Remote Sensing Images

• URL: http://arxiv.org/abs/2404.18583v2
• Date: Fri, 19 Jul 2024 11:46:17 GMT
• Title: Context Matters: Leveraging Spatiotemporal Metadata for Semi-Supervised Learning on Remote Sensing Images
• Authors: Maximilian Bernhard, Tanveer Hannan, Niklas Strauß, Matthias Schubert,
• Abstract summary: Current approaches generate pseudo-labels from model predictions for unlabeled samples. We propose exploiting totemporal metainformation in SSL to improve the quality of pseudo-labels. We show that adding the available metadata to the input of the predictor at test time degenerates the prediction quality for metadata outside thetemporal distribution of the training set.
• Score: 2.518656729567209
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Remote sensing projects typically generate large amounts of imagery that can be used to train powerful deep neural networks. However, the amount of labeled images is often small, as remote sensing applications generally require expert labelers. Thus, semi-supervised learning (SSL), i.e., learning with a small pool of labeled and a larger pool of unlabeled data, is particularly useful in this domain. Current SSL approaches generate pseudo-labels from model predictions for unlabeled samples. As the quality of these pseudo-labels is crucial for performance, utilizing additional information to improve pseudo-label quality yields a promising direction. For remote sensing images, geolocation and recording time are generally available and provide a valuable source of information as semantic concepts, such as land cover, are highly dependent on spatiotemporal context, e.g., due to seasonal effects and vegetation zones. In this paper, we propose to exploit spatiotemporal metainformation in SSL to improve the quality of pseudo-labels and, therefore, the final model performance. We show that directly adding the available metadata to the input of the predictor at test time degenerates the prediction quality for metadata outside the spatiotemporal distribution of the training set. Thus, we propose a teacher-student SSL framework where only the teacher network uses metainformation to improve the quality of pseudo-labels on the training set. Correspondingly, our student network benefits from the improved pseudo-labels but does not receive metadata as input, making it invariant to spatiotemporal shifts at test time. Furthermore, we propose methods for encoding and injecting spatiotemporal information into the model and introduce a novel distillation mechanism to enhance the knowledge transfer between teacher and student. Our framework dubbed Spatiotemporal SSL can be easily combined with several stat...

Related papers

• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix [59.55173022987071]
  We study the potential of semi-supervised learning for class-agnostic motion prediction. Our framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data. Our method exhibits comparable performance to weakly and some fully supervised methods.
  arXiv  Detail & Related papers  (2023-12-13T09:32:50Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• Synthetic Augmentation with Large-scale Unconditional Pre-training [4.162192894410251]
  We propose a synthetic augmentation method called HistoDiffusion to reduce the dependency on annotated data. HistoDiffusion can be pre-trained on large-scale unlabeled datasets and later applied to a small-scale labeled dataset for augmented training. We evaluate our proposed method by pre-training on three histopathology datasets and testing on a histopathology dataset of colorectal cancer (CRC) excluded from the pre-training datasets.
  arXiv  Detail & Related papers  (2023-08-08T03:34:04Z)
• Universal Semi-Supervised Learning for Medical Image Classification [21.781201758182135]
  Semi-supervised learning (SSL) has attracted much attention since it reduces the expensive costs of collecting adequate well-labeled training data. Traditional SSL is built upon an assumption that labeled and unlabeled data should be from the same distribution. We propose a unified framework to leverage unseen unlabeled data for open-scenario semi-supervised medical image classification.
  arXiv  Detail & Related papers  (2023-04-08T16:12:36Z)
• Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization [88.74813798138466]
  Localizing keypoints of an object is a basic visual problem. Supervised learning of a keypoint localization network often requires a large amount of data. We propose to automatically select reliable pseudo-labeled samples with a series of dynamic thresholds.
  arXiv  Detail & Related papers  (2022-01-21T09:51:58Z)
• Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images [0.2538209532048867]
  We propose a semi-weakly supervised contrastive learning framework for representation learning using semi-weakly annotated images. We empirically validate the transfer learning performance of SWCL on seven public retinal fundus datasets.
  arXiv  Detail & Related papers  (2021-08-04T15:50:09Z)
• Remote Sensing Image Scene Classification with Self-Supervised Paradigm under Limited Labeled Samples [11.025191332244919]
  We introduce new self-supervised learning (SSL) mechanism to obtain the high-performance pre-training model for RSIs scene classification from large unlabeled data. Experiments on three commonly used RSIs scene classification datasets demonstrated that this new learning paradigm outperforms the traditional dominant ImageNet pre-trained model. The insights distilled from our studies can help to foster the development of SSL in the remote sensing community.
  arXiv  Detail & Related papers  (2020-10-02T09:27:19Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)
• Density-Aware Graph for Deep Semi-Supervised Visual Recognition [102.9484812869054]
  Semi-supervised learning (SSL) has been extensively studied to improve the generalization ability of deep neural networks for visual recognition. This paper proposes to solve the SSL problem by building a novel density-aware graph, based on which the neighborhood information can be easily leveraged.
  arXiv  Detail & Related papers  (2020-03-30T02:52:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.