Fair contrastive pre-training for geographic image segmentation
- URL: http://arxiv.org/abs/2211.08672v2
- Date: Tue, 16 May 2023 01:15:04 GMT
- Title: Fair contrastive pre-training for geographic image segmentation
- Authors: Miao Zhang, Rumi Chunara
- Abstract summary: We show learnt representations present large performance gaps across selected sensitive groups.
We propose fair dense representations with contrastive learning to address the issue.
The method achieves improved downstream task fairness and outperforms state-of-the-art methods.
- Score: 31.576447346533225
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Contrastive self-supervised learning is widely employed in visual recognition
for geographic image data (remote or proximal sensing), but because of
landscape heterogeneity, models can show disparate performance across spatial
units. In this work, we consider fairness risks in such contrastive
pre-training; we show learnt representations present large performance gaps
across selected sensitive groups: urban and rural areas for satellite images
and city GDP level for street view images on downstream semantic segmentation.
We propose fair dense representations with contrastive learning (FairDCL) to
address the issue, a multi-level latent space de-biasing objective, using a
novel dense sensitive attribute encoding technique to constrain spurious local
information disparately distributes across groups. The method achieves improved
downstream task fairness and outperforms state-of-the-art methods for the
absence of a fairness-accuracy trade-off. Image embedding evaluation and
ablation studies further demonstrate effectiveness of FairDCL. As fairness in
geographic imagery is a nascent topic without existing state-of-the-art data or
results, our work motivates researchers to consider fairness metrics in such
applications, especially reinforced by our results showing no accuracy
degradation. Our code is available at:
https://anonymous.4open.science/r/FairDCL-1283
Related papers
- SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks.
To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
arXiv Detail & Related papers (2024-03-25T10:30:22Z) - Progressive Feature Self-reinforcement for Weakly Supervised Semantic
Segmentation [55.69128107473125]
We propose a single-stage approach for Weakly Supervised Semantic (WSSS) with image-level labels.
We adaptively partition the image content into deterministic regions (e.g., confident foreground and background) and uncertain regions (e.g., object boundaries and misclassified categories) for separate processing.
Building upon this, we introduce a complementary self-enhancement method that constrains the semantic consistency between these confident regions and an augmented image with the same class labels.
arXiv Detail & Related papers (2023-12-14T13:21:52Z) - Improving Fairness using Vision-Language Driven Image Augmentation [60.428157003498995]
Fairness is crucial when training a deep-learning discriminative model, especially in the facial domain.
Models tend to correlate specific characteristics (such as age and skin color) with unrelated attributes (downstream tasks)
This paper proposes a method to mitigate these correlations to improve fairness.
arXiv Detail & Related papers (2023-11-02T19:51:10Z) - Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
Fine-grained image recognition is a longstanding computer vision challenge.
We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem.
Our method significantly improves the generalization performance on several popular classification networks.
arXiv Detail & Related papers (2023-09-01T11:15:50Z) - CSP: Self-Supervised Contrastive Spatial Pre-Training for
Geospatial-Visual Representations [90.50864830038202]
We present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images.
We use a dual-encoder to separately encode the images and their corresponding geo-locations, and use contrastive objectives to learn effective location representations from images.
CSP significantly boosts the model performance with 10-34% relative improvement with various labeled training data sampling ratios.
arXiv Detail & Related papers (2023-05-01T23:11:18Z) - Segmenting across places: The need for fair transfer learning with
satellite imagery [24.087993065704527]
State-of-the-art models have better overall accuracy in rural areas compared to urban areas.
We show that raw satellite images are overall more dissimilar between source and target districts for rural than for urban locations.
arXiv Detail & Related papers (2022-04-09T02:14:56Z) - Fair Contrastive Learning for Facial Attribute Classification [25.436462696033846]
We propose a new Fair Supervised Contrastive Loss (FSCL) for fair visual representation learning.
In this paper, we for the first time analyze unfairness caused by supervised contrastive learning.
Our method is robust to the intensity of data bias and effectively works in incomplete supervised settings.
arXiv Detail & Related papers (2022-03-30T11:16:18Z) - Latent Space Smoothing for Individually Fair Representations [12.739528232133495]
We introduce LASSI, the first representation learning method for certifying individual fairness of high-dimensional data.
Our key insight is to leverage recent advances in generative modeling to capture the set of similar individuals in the generative latent space.
We employ randomized smoothing to provably map similar individuals close together, in turn ensuring that local robustness verification of the downstream application results in end-to-end fairness certification.
arXiv Detail & Related papers (2021-11-26T18:22:42Z) - Geography-Aware Self-Supervised Learning [79.4009241781968]
We show that due to their different characteristics, a non-trivial gap persists between contrastive and supervised learning on standard benchmarks.
We propose novel training methods that exploit the spatially aligned structure of remote sensing data.
Our experiments show that our proposed method closes the gap between contrastive and supervised learning on image classification, object detection and semantic segmentation for remote sensing.
arXiv Detail & Related papers (2020-11-19T17:29:13Z) - Representative-Discriminative Learning for Open-set Land Cover
Classification of Satellite Imagery [11.47389428456188]
We study the problem of open-set land cover classification that identifies the samples belonging to unknown classes during testing.
Although inherently a classification problem, both representative and discriminative aspects of data need to be exploited.
We propose a representative-discriminative open-set recognition framework.
arXiv Detail & Related papers (2020-07-21T15:28:56Z) - Self-supervising Fine-grained Region Similarities for Large-scale Image
Localization [43.1611420685653]
General public benchmarks only provide noisy GPS labels for learning image-to-image similarities.
We propose to self-supervise image-to-region similarities in order to fully explore the potential of difficult positive images alongside their sub-regions.
Our proposed self-enhanced image-to-region similarity labels effectively deal with the training bottleneck in the state-of-the-art pipelines.
arXiv Detail & Related papers (2020-06-06T17:31:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.