Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks

• URL: http://arxiv.org/abs/2112.01715v1
• Date: Fri, 3 Dec 2021 04:59:13 GMT
• Title: Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks
• Authors: Peri Akiva, Matthew Purri, Matthew Leotta
• Abstract summary: We present our material and texture based self-supervision method named MATTER (MATerial and TExture Representation Learning) MATerial and TExture Representation Learning is inspired by classical material and texture methods. We show that our self-supervision pre-training method allows for up to 24.22% and 6.33% performance increase in unsupervised and fine-tuned setups.
• Score: 5.5531367234797555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised learning aims to learn image feature representations without the usage of manually annotated labels. It is often used as a precursor step to obtain useful initial network weights which contribute to faster convergence and superior performance of downstream tasks. While self-supervision allows one to reduce the domain gap between supervised and unsupervised learning without the usage of labels, the self-supervised objective still requires a strong inductive bias to downstream tasks for effective transfer learning. In this work, we present our material and texture based self-supervision method named MATTER (MATerial and TExture Representation Learning), which is inspired by classical material and texture methods. Material and texture can effectively describe any surface, including its tactile properties, color, and specularity. By extension, effective representation of material and texture can describe other semantic classes strongly associated with said material and texture. MATTER leverages multi-temporal, spatially aligned remote sensing imagery over unchanged regions to learn invariance to illumination and viewing angle as a mechanism to achieve consistency of material and texture representation. We show that our self-supervision pre-training method allows for up to 24.22% and 6.33% performance increase in unsupervised and fine-tuned setups, and up to 76% faster convergence on change detection, land cover classification, and semantic segmentation tasks.

Related papers

• TIPS: Text-Image Pretraining with Spatial Awareness [13.38247732379754]
  Self-supervised image-only pretraining is still the go-to method for many vision applications. We propose a novel general-purpose image-text model, which can be effectively used off-the-shelf for dense and global vision tasks.
  arXiv  Detail & Related papers  (2024-10-21T21:05:04Z)
• Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction from LiDAR Data with Limited Annotations [1.3243401820948064]
  Building footprint maps offer promise of precise footprint extraction without extensive post-processing. Deep learning methods face challenges in generalization and label efficiency. We propose terrain-aware self-supervised learning tailored to remote sensing.
  arXiv  Detail & Related papers  (2023-11-02T12:34:23Z)
• Learning Transferable Pedestrian Representation from Multimodal Information Supervision [174.5150760804929]
  VAL-PAT is a novel framework that learns transferable representations to enhance various pedestrian analysis tasks with multimodal information. We first perform pre-training on LUPerson-TA dataset, where each image contains text and attribute annotations. We then transfer the learned representations to various downstream tasks, including person reID, person attribute recognition and text-based person search.
  arXiv  Detail & Related papers  (2023-04-12T01:20:58Z)
• Imposing Consistency for Optical Flow Estimation [73.53204596544472]
  Imposing consistency through proxy tasks has been shown to enhance data-driven learning. This paper introduces novel and effective consistency strategies for optical flow estimation.
  arXiv  Detail & Related papers  (2022-04-14T22:58:30Z)
• Semantic-Aware Generation for Self-Supervised Visual Representation Learning [116.5814634936371]
  We advocate for Semantic-aware Generation (SaGe) to facilitate richer semantics rather than details to be preserved in the generated image. SaGe complements the target network with view-specific features and thus alleviates the semantic degradation brought by intensive data augmentations. We execute SaGe on ImageNet-1K and evaluate the pre-trained models on five downstream tasks including nearest neighbor test, linear classification, and fine-scaled image recognition.
  arXiv  Detail & Related papers  (2021-11-25T16:46:13Z)
• Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting [168.91748514706995]
  We propose two novel cross-modal translation pre-text tasks for self-supervised feature learning: Vectorization and Rasterization. Our learned encoder modules benefit both-based and vector-based downstream approaches to analysing hand-drawn data.
  arXiv  Detail & Related papers  (2021-03-25T09:47:18Z)
• Can Semantic Labels Assist Self-Supervised Visual Representation Learning? [194.1681088693248]
  We present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN) In a series of downstream tasks, SCAN achieves superior performance compared to previous fully-supervised and self-supervised methods. Our study reveals that semantic labels are useful in assisting self-supervised methods, opening a new direction for the community.
  arXiv  Detail & Related papers  (2020-11-17T13:25:00Z)
• Learning Visual Representations for Transfer Learning by Suppressing Texture [38.901410057407766]
  In self-supervised learning, texture as a low-level cue may provide shortcuts that prevent the network from learning higher level representations. We propose to use classic methods based on anisotropic diffusion to augment training using images with suppressed texture. We empirically show that our method achieves state-of-the-art results on object detection and image classification.
  arXiv  Detail & Related papers  (2020-11-03T18:27:03Z)
• Semi-supervised Learning with a Teacher-student Network for Generalized Attribute Prediction [7.462336024223667]
  This paper presents a study on semi-supervised learning to solve the visual attribute prediction problem. Our method achieves competitive performance on various benchmarks for fashion attribute prediction.
  arXiv  Detail & Related papers  (2020-07-14T02:06:24Z)
• Learning Invariant Representations for Reinforcement Learning without Reconstruction [98.33235415273562]
  We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Bisimulation metrics quantify behavioral similarity between states in continuous MDPs. We demonstrate the effectiveness of our method at disregarding task-irrelevant information using modified visual MuJoCo tasks.
  arXiv  Detail & Related papers  (2020-06-18T17:59:35Z)
• Semantically-Guided Representation Learning for Self-Supervised Monocular Depth [40.49380547487908]
  We propose a new architecture leveraging fixed pretrained semantic segmentation networks to guide self-supervised representation learning. Our method improves upon the state of the art for self-supervised monocular depth prediction over all pixels, fine-grained details, and per semantic categories.
  arXiv  Detail & Related papers  (2020-02-27T18:40:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.