Fair contrastive pre-training for geographic image segmentation

• URL: http://arxiv.org/abs/2211.08672v2
• Date: Tue, 16 May 2023 01:15:04 GMT
• Title: Fair contrastive pre-training for geographic image segmentation
• Authors: Miao Zhang, Rumi Chunara
• Abstract summary: We show learnt representations present large performance gaps across selected sensitive groups. We propose fair dense representations with contrastive learning to address the issue. The method achieves improved downstream task fairness and outperforms state-of-the-art methods.
• Score: 31.576447346533225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive self-supervised learning is widely employed in visual recognition for geographic image data (remote or proximal sensing), but because of landscape heterogeneity, models can show disparate performance across spatial units. In this work, we consider fairness risks in such contrastive pre-training; we show learnt representations present large performance gaps across selected sensitive groups: urban and rural areas for satellite images and city GDP level for street view images on downstream semantic segmentation. We propose fair dense representations with contrastive learning (FairDCL) to address the issue, a multi-level latent space de-biasing objective, using a novel dense sensitive attribute encoding technique to constrain spurious local information disparately distributes across groups. The method achieves improved downstream task fairness and outperforms state-of-the-art methods for the absence of a fairness-accuracy trade-off. Image embedding evaluation and ablation studies further demonstrate effectiveness of FairDCL. As fairness in geographic imagery is a nascent topic without existing state-of-the-art data or results, our work motivates researchers to consider fairness metrics in such applications, especially reinforced by our results showing no accuracy degradation. Our code is available at: https://anonymous.4open.science/r/FairDCL-1283

Related papers

• Object Affordance Recognition and Grounding via Multi-scale Cross-modal Representation Learning [64.32618490065117]
  A core problem of Embodied AI is to learn object manipulation from observation, as humans do.<n>We propose a novel approach that learns an affordance-aware 3D representation and employs a stage-wise inference strategy.<n> Experiments demonstrate the effectiveness of our method, showing improved performance in both affordance grounding and classification.
  arXiv  Detail & Related papers  (2025-08-02T04:14:18Z)
• AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization [57.34659640776723]
  We propose an end-to-end framework named AddressCLIP to solve the problem with more semantics. We have built three datasets from Pittsburgh and San Francisco on different scales specifically for the IAL problem.
  arXiv  Detail & Related papers  (2024-07-11T03:18:53Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• Region-aware Distribution Contrast: A Novel Approach to Multi-Task Partially Supervised Learning [50.88504784466931]
  Multi-task dense prediction involves semantic segmentation, depth estimation, and surface normal estimation. Existing solutions typically rely on learning global image representations for global cross-task image matching. Our proposal involves modeling region-wise representations using Gaussian Distributions.
  arXiv  Detail & Related papers  (2024-03-15T12:41:30Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Segmenting across places: The need for fair transfer learning with satellite imagery [24.087993065704527]
  State-of-the-art models have better overall accuracy in rural areas compared to urban areas. We show that raw satellite images are overall more dissimilar between source and target districts for rural than for urban locations.
  arXiv  Detail & Related papers  (2022-04-09T02:14:56Z)
• Semi-Supervised Contrastive Learning for Remote Sensing: Identifying Ancient Urbanization in the South Central Andes [11.489739686646647]
  In this study, we use 95,358 unlabeled images and 5,830 labelled images in order to solve the issues associated with detecting ancient buildings from a long-tailed satellite image dataset. Our semi-supervised contrastive learning model achieved a promising testing balanced accuracy of 79.0%, which is a 3.8% improvement as compared to other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-12-13T06:26:47Z)
• Spatially Correlated Patterns in Adversarial Images [5.069312274160184]
  Adversarial attacks have proved to be the major impediment in the progress on research towards reliable machine learning solutions. We propose a framework for segregating and isolating regions within an input image which are critical towards either classification (during inference), or adversarial vulnerability or both.
  arXiv  Detail & Related papers  (2020-11-21T14:06:59Z)
• Adversarial Semantic Data Augmentation for Human Pose Estimation [96.75411357541438]
  We propose Semantic Data Augmentation (SDA), a method that augments images by pasting segmented body parts with various semantic granularity. We also propose Adversarial Semantic Data Augmentation (ASDA), which exploits a generative network to dynamiclly predict tailored pasting configuration. State-of-the-art results are achieved on challenging benchmarks.
  arXiv  Detail & Related papers  (2020-08-03T07:56:04Z)
• Representative-Discriminative Learning for Open-set Land Cover Classification of Satellite Imagery [11.47389428456188]
  We study the problem of open-set land cover classification that identifies the samples belonging to unknown classes during testing. Although inherently a classification problem, both representative and discriminative aspects of data need to be exploited. We propose a representative-discriminative open-set recognition framework.
  arXiv  Detail & Related papers  (2020-07-21T15:28:56Z)
• Self-supervising Fine-grained Region Similarities for Large-scale Image Localization [43.1611420685653]
  General public benchmarks only provide noisy GPS labels for learning image-to-image similarities. We propose to self-supervise image-to-region similarities in order to fully explore the potential of difficult positive images alongside their sub-regions. Our proposed self-enhanced image-to-region similarity labels effectively deal with the training bottleneck in the state-of-the-art pipelines.
  arXiv  Detail & Related papers  (2020-06-06T17:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.