Annotation Cost-Efficient Active Learning for Deep Metric Learning Driven Remote Sensing Image Retrieval
- URL: http://arxiv.org/abs/2406.10107v1
- Date: Fri, 14 Jun 2024 15:08:04 GMT
- Title: Annotation Cost-Efficient Active Learning for Deep Metric Learning Driven Remote Sensing Image Retrieval
- Authors: Genc Hoxha, Gencer Sumbul, Julia Henkel, Lars Möllenbrok, Begüm Demir,
- Abstract summary: ANNEAL aims to create a small but informative training set made up of similar and dissimilar image pairs.
The selected image pairs are sent to expert annotators to be labeled as similar or dissimilar.
This way of annotating images significantly reduces the annotation cost compared to the cost of annotating images with LULC labels.
- Score: 3.2109665109975696
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Deep metric learning (DML) has shown to be very effective for content-based image retrieval (CBIR) in remote sensing (RS). Most of DML methods for CBIR rely on many annotated images to accurately learn model parameters of deep neural networks. However, gathering many image annotations is time consuming and costly. To address this, we propose an annotation cost-efficient active learning (ANNEAL) method specifically designed for DML driven CBIR in RS. ANNEAL aims to create a small but informative training set made up of similar and dissimilar image pairs to be utilized for learning a deep metric space. The informativeness of the image pairs is assessed combining uncertainty and diversity criteria. To assess the uncertainty of image pairs, we introduce two algorithms: 1) metric-guided uncertainty estimation (MGUE); and 2) binary classifier guided uncertainty estimation (BCGUE). MGUE automatically estimates a threshold value that acts as a "boundary" between similar and dissimilar image pairs based on the distances in the metric space. The closer the similarity between image pairs to the estimated threshold value the higher their uncertainty. BCGUE estimates the uncertainty of the image pairs based on the confidence of the classifier in assigning the correct similarity label. The diversity criterion is assessed through a clustering-based strategy. ANNEAL selects the most informative image pairs by combining either MGUE or BCGUE with clustering-based strategy. The selected image pairs are sent to expert annotators to be labeled as similar or dissimilar. This way of annotating images significantly reduces the annotation cost compared to the cost of annotating images with LULC labels. Experimental results carried out on two RS benchmark datasets demonstrate the effectiveness of our method. The code of the proposed method will be publicly available upon the acceptance of the paper.
Related papers
- Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based
Image Retrieval [69.46139774646308]
This paper studies the problem of zero-shot sketch-based image retrieval (ZS-SBIR)
It aims to use sketches from unseen categories as queries to match the images of the same category.
We propose a novel Symmetrical Bidirectional Knowledge Alignment for zero-shot sketch-based image retrieval (SBKA)
arXiv Detail & Related papers (2023-12-16T04:50:34Z) - Introspective Deep Metric Learning [91.47907685364036]
We propose an introspective deep metric learning framework for uncertainty-aware comparisons of images.
The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling.
arXiv Detail & Related papers (2023-09-11T16:21:13Z) - Improving Human-Object Interaction Detection via Virtual Image Learning [68.56682347374422]
Human-Object Interaction (HOI) detection aims to understand the interactions between humans and objects.
In this paper, we propose to alleviate the impact of such an unbalanced distribution via Virtual Image Leaning (VIL)
A novel label-to-image approach, Multiple Steps Image Creation (MUSIC), is proposed to create a high-quality dataset that has a consistent distribution with real images.
arXiv Detail & Related papers (2023-08-04T10:28:48Z) - Annotation Cost Efficient Active Learning for Content Based Image
Retrieval [1.6624384368855525]
We present an annotation cost efficient active learning (AL) method (denoted as ANNEAL)
The proposed method aims to iteratively enrich the training set by annotating the most informative image pairs as similar or dissimilar.
The code of ANNEAL is publicly available at https://git.tu-berlin.de/rsim/ANNEAL.
arXiv Detail & Related papers (2023-06-20T15:33:24Z) - Introspective Deep Metric Learning for Image Retrieval [80.29866561553483]
We argue that a good similarity model should consider the semantic discrepancies with caution to better deal with ambiguous images for more robust training.
We propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively.
The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets.
arXiv Detail & Related papers (2022-05-09T17:51:44Z) - Deep Relational Metric Learning [84.95793654872399]
This paper presents a deep relational metric learning framework for image clustering and retrieval.
We learn an ensemble of features that characterizes an image from different aspects to model both interclass and intraclass distributions.
Experiments on the widely-used CUB-200-2011, Cars196, and Stanford Online Products datasets demonstrate that our framework improves existing deep metric learning methods and achieves very competitive results.
arXiv Detail & Related papers (2021-08-23T09:31:18Z) - A Novel Triplet Sampling Method for Multi-Label Remote Sensing Image
Search and Retrieval [1.123376893295777]
A common approach for learning the metric space relies on the selection of triplets of similar (positive) and dissimilar (negative) images.
We propose a novel triplet sampling method in the framework of deep neural networks (DNNs) defined for multi-label RS CBIR problems.
arXiv Detail & Related papers (2021-05-08T09:16:09Z) - Learning to Focus: Cascaded Feature Matching Network for Few-shot Image
Recognition [38.49419948988415]
Deep networks can learn to accurately recognize objects of a category by training on a large number of images.
A meta-learning challenge known as a low-shot image recognition task comes when only a few images with annotations are available for learning a recognition model for one category.
Our method, called Cascaded Feature Matching Network (CFMN), is proposed to solve this problem.
Experiments for few-shot learning on two standard datasets, emphminiImageNet and Omniglot, have confirmed the effectiveness of our method.
arXiv Detail & Related papers (2021-01-13T11:37:28Z) - DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning [122.51237307910878]
We develop methods for few-shot image classification from a new perspective of optimal matching between image regions.
We employ the Earth Mover's Distance (EMD) as a metric to compute a structural distance between dense image representations.
To generate the important weights of elements in the formulation, we design a cross-reference mechanism.
arXiv Detail & Related papers (2020-03-15T08:13:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.