No Annotations for Object Detection in Art through Stable Diffusion

• URL: http://arxiv.org/abs/2412.06286v2
• Date: Wed, 18 Dec 2024 02:44:01 GMT
• Title: No Annotations for Object Detection in Art through Stable Diffusion
• Authors: Patrick Ramos, Nicolas Gonthier, Selina Khan, Yuta Nakashima, Noa Garcia,
• Abstract summary: We present NADA (no annotations for detection in art), a pipeline that leverages diffusion models' art-related knowledge for object detection in paintings without the need for full bounding box supervision.<n>Our method supports both weakly-supervised and zero-shot scenarios and does not require any fine-tuning of its pretrained components.
• Score: 21.755357781998853
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object detection in art is a valuable tool for the digital humanities, as it allows for faster identification of objects in artistic and historical images compared to humans. However, annotating such images poses significant challenges due to the need for specialized domain expertise. We present NADA (no annotations for detection in art), a pipeline that leverages diffusion models' art-related knowledge for object detection in paintings without the need for full bounding box supervision. Our method, which supports both weakly-supervised and zero-shot scenarios and does not require any fine-tuning of its pretrained components, consists of a class proposer based on large vision-language models and a class-conditioned detector based on Stable Diffusion. NADA is evaluated on two artwork datasets, ArtDL 2.0 and IconArt, outperforming prior work in weakly-supervised detection, while being the first work for zero-shot object detection in art. Code is available at https://github.com/patrick-john-ramos/nada

Related papers

• Exploiting Unlabeled Data with Multiple Expert Teachers for Open Vocabulary Aerial Object Detection and Its Orientation Adaptation [58.37525311718006]
  We put forth a novel formulation of the aerial object detection problem, namely open-vocabulary aerial object detection (OVAD) We propose CastDet, a CLIP-activated student-teacher detection framework that serves as the first OVAD detector specifically designed for the challenging aerial scenario. Our framework integrates a robust localization teacher along with several box selection strategies to generate high-quality proposals for novel objects.
  arXiv  Detail & Related papers  (2024-11-04T12:59:13Z)
• PEEKABOO: Hiding parts of an image for unsupervised object localization [7.161489957025654]
  Localizing objects in an unsupervised manner poses significant challenges due to the absence of key visual information. We propose a single-stage learning framework, dubbed PEEKABOO, for unsupervised object localization. The key idea is to selectively hide parts of an image and leverage the remaining image information to infer the location of objects without explicit supervision.
  arXiv  Detail & Related papers  (2024-07-24T20:35:20Z)
• Few-shot Oriented Object Detection with Memorable Contrastive Learning in Remote Sensing Images [11.217630579076237]
  Few-shot object detection (FSOD) has garnered significant research attention in the field of remote sensing. We propose a novel FSOD method for remote sensing images called Few-shot Oriented object detection with Memorable Contrastive learning (FOMC) Specifically, we employ oriented bounding boxes instead of traditional horizontal bounding boxes to learn a better feature representation for arbitrary-oriented aerial objects.
  arXiv  Detail & Related papers  (2024-03-20T08:15:18Z)
• Exploiting Unlabeled Data with Vision and Language Models for Object Detection [64.94365501586118]
  Building robust and generic object detection frameworks requires scaling to larger label spaces and bigger training datasets. We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images. We demonstrate the value of the generated pseudo labels in two specific tasks, open-vocabulary detection and semi-supervised object detection.
  arXiv  Detail & Related papers  (2022-07-18T21:47:15Z)
• Learning to Detect Every Thing in an Open World [139.78830329914135]
  We propose a simple yet surprisingly powerful data augmentation and training scheme we call Learning to Detect Every Thing (LDET) To avoid suppressing hidden objects, background objects that are visible but unlabeled, we paste annotated objects on a background image sampled from a small region of the original image. LDET leads to significant improvements on many datasets in the open world instance segmentation task.
  arXiv  Detail & Related papers  (2021-12-03T03:56:06Z)
• A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection [56.82077636126353]
  We take advantage of object-centric images to improve object detection in scene-centric images. We present a simple yet surprisingly effective framework to do so. Our approach can improve the object detection (and instance segmentation) accuracy of rare objects by 50% (and 33%) relatively.
  arXiv  Detail & Related papers  (2021-02-17T17:27:21Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)
• Learning Object Detection from Captions via Textual Scene Attributes [70.90708863394902]
  We argue that captions contain much richer information about the image, including attributes of objects and their relations. We present a method that uses the attributes in this "textual scene graph" to train object detectors. We empirically demonstrate that the resulting model achieves state-of-the-art results on several challenging object detection datasets.
  arXiv  Detail & Related papers  (2020-09-30T10:59:20Z)
• A Self-supervised GAN for Unsupervised Few-shot Object Recognition [39.79912546252623]
  This paper addresses unsupervised few-shot object recognition. All training images are unlabeled, and test images are divided into queries and a few labeled support images per object class of interest. We extend the vanilla GAN with two loss functions, both aimed at self-supervised learning.
  arXiv  Detail & Related papers  (2020-08-16T19:47:26Z)
• Co-training for On-board Deep Object Detection [0.0]
  Best performing deep vision-based object detectors are trained in a supervised manner by relying on human-labeled bounding boxes. Co-training is a semi-supervised learning method for self-labeling objects in unlabeled images. We show how co-training is a paradigm worth to pursue for alleviating object labeling, working both alone and together with task-agnostic domain adaptation.
  arXiv  Detail & Related papers  (2020-08-12T19:08:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.