SLANT: Spurious Logo ANalysis Toolkit

• URL: http://arxiv.org/abs/2406.01449v1
• Date: Mon, 3 Jun 2024 15:41:31 GMT
• Title: SLANT: Spurious Logo ANalysis Toolkit
• Authors: Maan Qraitem, Piotr Teterwak, Kate Saenko, Bryan A. Plummer,
• Abstract summary: We develop SLANT: A Spurious Logo ANalysis Toolkit. It contains a semi-automatic mechanism for mining such "spurious" logos. We uncover various seemingly harmless logos that VL models correlate with negative human adjectives. An attacker could place a spurious logo on harmful content, causing the model to misclassify it as harmless.
• Score: 61.59021920232986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online content is filled with logos, from ads and social media posts to website branding and product placements. Consequently, these logos are prevalent in the extensive web-scraped datasets used to pretrain Vision-Language Models, which are used for a wide array of tasks (content moderation, object classification). While these models have been shown to learn harmful correlations in various tasks, whether these correlations include logos remains understudied. Understanding this is especially important due to logos often being used by public-facing entities like brands and government agencies. To that end, we develop SLANT: A Spurious Logo ANalysis Toolkit. Our key finding is that some logos indeed lead to spurious incorrect predictions, for example, adding the Adidas logo to a photo of a person causes a model classify the person as greedy. SLANT contains a semi-automatic mechanism for mining such "spurious" logos. The mechanism consists of a comprehensive logo bank, CC12M-LogoBank, and an algorithm that searches the bank for logos that VLMs spuriously correlate with a user-provided downstream recognition target. We uncover various seemingly harmless logos that VL models correlate 1) with negative human adjectives 2) with the concept of `harmlessness'; causing models to misclassify harmful online content as harmless, and 3) with user-provided object concepts; causing lower recognition accuracy on ImageNet zero-shot classification. Furthermore, SLANT's logos can be seen as effective attacks against foundational models; an attacker could place a spurious logo on harmful content, causing the model to misclassify it as harmless. This threat is alarming considering the simplicity of logo attacks, increasing the attack surface of VL models. As a defense, we include in our Toolkit two effective mitigation strategies that seamlessly integrate with zero-shot inference of foundation models.

Related papers

• LogoSticker: Inserting Logos into Diffusion Models for Customized Generation [73.59571559978278]
  We introduce the task of logo insertion into text-to-image models. Our goal is to insert logo identities into diffusion models and enable their seamless synthesis in varied contexts. We present a novel two-phase pipeline LogoSticker to tackle this task.
  arXiv  Detail & Related papers  (2024-07-18T17:54:49Z)
• LogoNet: a fine-grained network for instance-level logo sketch retrieval [7.858317554338922]
  We construct an instance-level logo sketch dataset containing 2k logo instances and exceeding 9k sketches. Next, we develop a fine-grained triple-branch CNN architecture based on hybrid attention mechanism termed LogoNet for accurate logo sketch retrieval.
  arXiv  Detail & Related papers  (2023-04-05T04:03:02Z)
• Contrastive Multi-View Textual-Visual Encoding: Towards One Hundred Thousand-Scale One-Shot Logo Identification [2.243832625209014]
  We study the problem of identifying logos of business brands in natural scenes in an open-set one-shot setting. We propose a novel multi-view textual-visual encoding framework that encodes text appearing in the logos. We evaluate our proposed framework for cropped logo verification, cropped logo identification, and end-to-end logo identification in natural scene tasks.
  arXiv  Detail & Related papers  (2022-11-23T12:59:41Z)
• Discriminative Semantic Feature Pyramid Network with Guided Anchoring for Logo Detection [52.36825190893928]
  We propose a novel approach, named Discriminative Semantic Feature Pyramid Network with Guided Anchoring (DSFP-GA) Our approach mainly consists of Discriminative Semantic Feature Pyramid (DSFP) and Guided Anchoring (GA)
  arXiv  Detail & Related papers  (2021-08-31T11:59:00Z)
• An Effective and Robust Detector for Logo Detection [58.448716977297565]
  Some attackers fool the well-trained logo detection model for infringement. A novel logo detector based on the mechanism of looking and thinking twice is proposed in this paper. We extend detectoRS algorithm to a cascade schema with an equalization loss function, multi-scale transformations, and adversarial data augmentation.
  arXiv  Detail & Related papers  (2021-08-01T10:17:53Z)
• Attack to Fool and Explain Deep Networks [59.97135687719244]
  We counter-argue by providing evidence of human-meaningful patterns in adversarial perturbations. Our major contribution is a novel pragmatic adversarial attack that is subsequently transformed into a tool to interpret the visual models.
  arXiv  Detail & Related papers  (2021-06-20T03:07:36Z)
• LogoDet-3K: A Large-Scale Image Dataset for Logo Detection [61.296935298332606]
  We introduce LogoDet-3K, the largest logo detection dataset with full annotation. It has 3,000 logo categories, about 200,000 manually annotated logo objects and 158,652 images. We propose a strong baseline method Logo-Yolo, which incorporates Focal loss and CIoU loss into the state-of-the-art YOLOv3 framework for large-scale logo detection.
  arXiv  Detail & Related papers  (2020-08-12T14:57:53Z)
• Network of Steel: Neural Font Style Transfer from Heavy Metal to Corporate Logos [0.18275108630751835]
  We introduce a method for transferring style from the logos of heavy metal bands onto corporate logos using a VGG16 network. We establish the contribution of different layers and loss coefficients to the learning of style. We find layers and loss coefficients that produce a good tradeoff between heavy metal style and corporate logo readability.
  arXiv  Detail & Related papers  (2020-01-10T20:41:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.