CorrEmbed: Evaluating Pre-trained Model Image Similarity Efficacy with a Novel Metric

• URL: http://arxiv.org/abs/2308.16126v1
• Date: Wed, 30 Aug 2023 16:23:07 GMT
• Title: CorrEmbed: Evaluating Pre-trained Model Image Similarity Efficacy with a Novel Metric
• Authors: Karl Audun Kagnes Borgersen, Morten Goodwin, Jivitesh Sharma, Tobias Aasmoe, Mari Leonhardsen, Gro Herredsvela R{\o}rvik
• Abstract summary: We evaluate the viability of the image embeddings from pre-trained computer vision models using a novel approach named CorrEmbed. Our approach computes the correlation between distances in image embeddings and distances in human-generated tag vectors. Our method also identifies deviations from this pattern, providing insights into how different models capture high-level image features.
• Score: 6.904776368895614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting visually similar images is a particularly useful attribute to look to when calculating product recommendations. Embedding similarity, which utilizes pre-trained computer vision models to extract high-level image features, has demonstrated remarkable efficacy in identifying images with similar compositions. However, there is a lack of methods for evaluating the embeddings generated by these models, as conventional loss and performance metrics do not adequately capture their performance in image similarity search tasks. In this paper, we evaluate the viability of the image embeddings from numerous pre-trained computer vision models using a novel approach named CorrEmbed. Our approach computes the correlation between distances in image embeddings and distances in human-generated tag vectors. We extensively evaluate numerous pre-trained Torchvision models using this metric, revealing an intuitive relationship of linear scaling between ImageNet1k accuracy scores and tag-correlation scores. Importantly, our method also identifies deviations from this pattern, providing insights into how different models capture high-level image features. By offering a robust performance evaluation of these pre-trained models, CorrEmbed serves as a valuable tool for researchers and practitioners seeking to develop effective, data-driven approaches to similar item recommendations in fashion retail.

Related papers

• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Image Similarity using An Ensemble of Context-Sensitive Models [2.9490616593440317]
  We present a more intuitive approach to build and compare image similarity models based on labelled data. We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data. Our testing results show that the ensemble model constructed performs 5% better than the best individual context-sensitive models.
  arXiv  Detail & Related papers  (2024-01-15T20:23:05Z)
• Scaling Laws of Synthetic Images for Model Training ... for Now [54.43596959598466]
  We study the scaling laws of synthetic images generated by state of the art text-to-image models. We observe that synthetic images demonstrate a scaling trend similar to, but slightly less effective than, real images in CLIP training.
  arXiv  Detail & Related papers  (2023-12-07T18:59:59Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• Effective Robustness against Natural Distribution Shifts for Models with Different Training Data [113.21868839569]
  "Effective robustness" measures the extra out-of-distribution robustness beyond what can be predicted from the in-distribution (ID) performance. We propose a new evaluation metric to evaluate and compare the effective robustness of models trained on different data.
  arXiv  Detail & Related papers  (2023-02-02T19:28:41Z)
• Learning an Adaptation Function to Assess Image Visual Similarities [0.0]
  We focus here on the specific task of learning visual image similarities when analogy matters. We propose to compare different supervised, semi-supervised and self-supervised networks, pre-trained on distinct scales and contents datasets. Our experiments conducted on the Totally Looks Like image dataset highlight the interest of our method, by increasing the retrieval scores of the best model @1 by 2.25x.
  arXiv  Detail & Related papers  (2022-06-03T07:15:00Z)
• Evaluating Contrastive Models for Instance-based Image Retrieval [6.393147386784114]
  We evaluate contrastive models for the task of image retrieval. We find that models trained using contrastive methods perform on-par with (and outperforms) a pre-trained baseline trained on the ImageNet labels.
  arXiv  Detail & Related papers  (2021-04-30T12:05:23Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• An application of a pseudo-parabolic modeling to texture image recognition [0.0]
  We present a novel methodology for texture image recognition using a partial differential equation modeling. We employ the pseudo-parabolic Buckley-Leverett equation to provide a dynamics to the digital image representation and collect local descriptors from those images evolving in time.
  arXiv  Detail & Related papers  (2021-02-09T18:08:42Z)
• Using Wavelets to Analyze Similarities in Image-Classification Datasets [0.0]
  We develop a practical and fast method to analyze the similarities in image classification datasets. Our method uses wavelet decomposition of images and other numerical analysis tools, with no need for a pre-trained model. We show that similar images in standard datasets can be identified in a few seconds, a significant speed-up compared to alternative methods in the literature.
  arXiv  Detail & Related papers  (2020-02-24T13:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.