On Background Bias in Deep Metric Learning

• URL: http://arxiv.org/abs/2210.01615v1
• Date: Tue, 4 Oct 2022 13:57:39 GMT
• Title: On Background Bias in Deep Metric Learning
• Authors: Konstantin Kobs and Andreas Hotho
• Abstract summary: We analyze the influence of the image background on Deep Metric Learning models. We show that replacing the background of images during training with random background images alleviates this issue.
• Score: 5.368313160283353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Metric Learning trains a neural network to map input images to a lower-dimensional embedding space such that similar images are closer together than dissimilar images. When used for item retrieval, a query image is embedded using the trained model and the closest items from a database storing their respective embeddings are returned as the most similar items for the query. Especially in product retrieval, where a user searches for a certain product by taking a photo of it, the image background is usually not important and thus should not influence the embedding process. Ideally, the retrieval process always returns fitting items for the photographed object, regardless of the environment the photo was taken in. In this paper, we analyze the influence of the image background on Deep Metric Learning models by utilizing five common loss functions and three common datasets. We find that Deep Metric Learning networks are prone to so-called background bias, which can lead to a severe decrease in retrieval performance when changing the image background during inference. We also show that replacing the background of images during training with random background images alleviates this issue. Since we use an automatic background removal method to do this background replacement, no additional manual labeling work and model changes are required while inference time stays the same. Qualitative and quantitative analyses, for which we introduce a new evaluation metric, confirm that models trained with replaced backgrounds attend more to the main object in the image, benefitting item retrieval systems.

Related papers

• Identifying Bias in Deep Neural Networks Using Image Transforms [0.0]
  CNNs have become one of the most commonly used computational tools in the past two decades. One of the primary downsides of CNNs is that they work as a black box', where the user cannot necessarily know how the image data are analyzed. This can lead to hidden biases that affect the performance evaluation of neural networks, but are difficult to identify.
  arXiv  Detail & Related papers  (2024-12-17T16:51:44Z)
• Targeted Background Removal Creates Interpretable Feature Visualizations [0.0]
  We argue that by using background removal techniques as a form of robust training, a network is forced to learn more human recognizable features. Four different training methods were used to verify this hypothesis. The feature visualization results show that the background removed images reveal a significant improvement over the baseline model.
  arXiv  Detail & Related papers  (2023-06-22T19:39:06Z)
• Query Semantic Reconstruction for Background in Few-Shot Segmentation [0.0]
  Few-shot segmentation (FSS) aims to segment unseen classes using a few annotated samples. Some FSS methods try to address this issue by using the background in the support image(s) to help identify the background in the query image. This article proposes a method, QSR, that extracts the background from the query image itself.
  arXiv  Detail & Related papers  (2022-10-21T15:49:16Z)
• Fusing Local Similarities for Retrieval-based 3D Orientation Estimation of Unseen Objects [70.49392581592089]
  We tackle the task of estimating the 3D orientation of previously-unseen objects from monocular images. We follow a retrieval-based strategy and prevent the network from learning object-specific features. Our experiments on the LineMOD, LineMOD-Occluded, and T-LESS datasets show that our method yields a significantly better generalization to unseen objects than previous works.
  arXiv  Detail & Related papers  (2022-03-16T08:53:00Z)
• Object-aware Contrastive Learning for Debiased Scene Representation [74.30741492814327]
  We develop a novel object-aware contrastive learning framework that localizes objects in a self-supervised manner. We also introduce two data augmentations based on ContraCAM, object-aware random crop and background mixup, which reduce contextual and background biases during contrastive self-supervised learning.
  arXiv  Detail & Related papers  (2021-07-30T19:24:07Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• On the Unreasonable Effectiveness of Centroids in Image Retrieval [0.1933681537640272]
  We propose to use the mean centroid representation both during training and retrieval. As each class is represented by a single embedding - the class centroid - both retrieval time and storage requirements are reduced significantly.
  arXiv  Detail & Related papers  (2021-04-28T08:57:57Z)
• Scale Normalized Image Pyramids with AutoFocus for Object Detection [75.71320993452372]
  A scale normalized image pyramid (SNIP) is generated that, like human vision, only attends to objects within a fixed size range at different scales. We propose an efficient spatial sub-sampling scheme which only operates on fixed-size sub-regions likely to contain objects. The resulting algorithm is referred to as AutoFocus and results in a 2.5-5 times speed-up during inference when used with SNIP.
  arXiv  Detail & Related papers  (2021-02-10T18:57:53Z)
• Noise or Signal: The Role of Image Backgrounds in Object Recognition [93.55720207356603]
  We create a toolkit for disentangling foreground and background signal on ImageNet images. We find that (a) models can achieve non-trivial accuracy by relying on the background alone, (b) models often misclassify images even in the presence of correctly classified foregrounds.
  arXiv  Detail & Related papers  (2020-06-17T16:54:43Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.