Mitigating Urban-Rural Disparities in Contrastive Representation Learning with Satellite Imagery

• URL: http://arxiv.org/abs/2211.08672v3
• Date: Wed, 18 Sep 2024 14:35:56 GMT
• Title: Mitigating Urban-Rural Disparities in Contrastive Representation Learning with Satellite Imagery
• Authors: Miao Zhang, Rumi Chunara,
• Abstract summary: We consider the risk of urban-rural disparities in identification of land-cover features. We propose fair dense representation with contrastive learning (FairDCL) as a method for de-biasing the multi-level latent space of convolution neural network models. The obtained image representation mitigates downstream urban-rural prediction disparities and outperforms state-of-the-art baselines on real-world satellite images.
• Score: 19.93324644519412
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Satellite imagery is being leveraged for many societally critical tasks across climate, economics, and public health. Yet, because of heterogeneity in landscapes (e.g. how a road looks in different places), models can show disparate performance across geographic areas. Given the important potential of disparities in algorithmic systems used in societal contexts, here we consider the risk of urban-rural disparities in identification of land-cover features. This is via semantic segmentation (a common computer vision task in which image regions are labelled according to what is being shown) which uses pre-trained image representations generated via contrastive self-supervised learning. We propose fair dense representation with contrastive learning (FairDCL) as a method for de-biasing the multi-level latent space of convolution neural network models. The method improves feature identification by removing spurious model representations which are disparately distributed across urban and rural areas, and is achieved in an unsupervised way by contrastive pre-training. The obtained image representation mitigates downstream urban-rural prediction disparities and outperforms state-of-the-art baselines on real-world satellite images. Embedding space evaluation and ablation studies further demonstrate FairDCL's robustness. As generalizability and robustness in geographic imagery is a nascent topic, our work motivates researchers to consider metrics beyond average accuracy in such applications.

Related papers

• AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization [57.34659640776723]
  We propose an end-to-end framework named AddressCLIP to solve the problem with more semantics. We have built three datasets from Pittsburgh and San Francisco on different scales specifically for the IAL problem.
  arXiv  Detail & Related papers  (2024-07-11T03:18:53Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• Region-aware Distribution Contrast: A Novel Approach to Multi-Task Partially Supervised Learning [50.88504784466931]
  Multi-task dense prediction involves semantic segmentation, depth estimation, and surface normal estimation. Existing solutions typically rely on learning global image representations for global cross-task image matching. Our proposal involves modeling region-wise representations using Gaussian Distributions.
  arXiv  Detail & Related papers  (2024-03-15T12:41:30Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Segmenting across places: The need for fair transfer learning with satellite imagery [24.087993065704527]
  State-of-the-art models have better overall accuracy in rural areas compared to urban areas. We show that raw satellite images are overall more dissimilar between source and target districts for rural than for urban locations.
  arXiv  Detail & Related papers  (2022-04-09T02:14:56Z)
• Semi-Supervised Contrastive Learning for Remote Sensing: Identifying Ancient Urbanization in the South Central Andes [11.489739686646647]
  In this study, we use 95,358 unlabeled images and 5,830 labelled images in order to solve the issues associated with detecting ancient buildings from a long-tailed satellite image dataset. Our semi-supervised contrastive learning model achieved a promising testing balanced accuracy of 79.0%, which is a 3.8% improvement as compared to other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-12-13T06:26:47Z)
• Spatially Correlated Patterns in Adversarial Images [5.069312274160184]
  Adversarial attacks have proved to be the major impediment in the progress on research towards reliable machine learning solutions. We propose a framework for segregating and isolating regions within an input image which are critical towards either classification (during inference), or adversarial vulnerability or both.
  arXiv  Detail & Related papers  (2020-11-21T14:06:59Z)
• Adversarial Semantic Data Augmentation for Human Pose Estimation [96.75411357541438]
  We propose Semantic Data Augmentation (SDA), a method that augments images by pasting segmented body parts with various semantic granularity. We also propose Adversarial Semantic Data Augmentation (ASDA), which exploits a generative network to dynamiclly predict tailored pasting configuration. State-of-the-art results are achieved on challenging benchmarks.
  arXiv  Detail & Related papers  (2020-08-03T07:56:04Z)
• Representative-Discriminative Learning for Open-set Land Cover Classification of Satellite Imagery [11.47389428456188]
  We study the problem of open-set land cover classification that identifies the samples belonging to unknown classes during testing. Although inherently a classification problem, both representative and discriminative aspects of data need to be exploited. We propose a representative-discriminative open-set recognition framework.
  arXiv  Detail & Related papers  (2020-07-21T15:28:56Z)
• Self-supervising Fine-grained Region Similarities for Large-scale Image Localization [43.1611420685653]
  General public benchmarks only provide noisy GPS labels for learning image-to-image similarities. We propose to self-supervise image-to-region similarities in order to fully explore the potential of difficult positive images alongside their sub-regions. Our proposed self-enhanced image-to-region similarity labels effectively deal with the training bottleneck in the state-of-the-art pipelines.
  arXiv  Detail & Related papers  (2020-06-06T17:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.