Detect-and-describe: Joint learning framework for detection and description of objects

• URL: http://arxiv.org/abs/2204.08828v1
• Date: Tue, 19 Apr 2022 11:57:30 GMT
• Title: Detect-and-describe: Joint learning framework for detection and description of objects
• Authors: Addel Zafar, Umar Khalid
• Abstract summary: We present a new approach to simultaneously detect objects and infer their attributes, we call it Detect and Describe (DaD) framework. DaD is a deep learning-based approach that extends object detection to object attribute prediction as well. We achieve 97.0% in Area Under the Receiver Operating Characteristic Curve (AUC) for object attributes prediction on aPascal test set.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional object detection answers two questions; "what" (what the object is?) and "where" (where the object is?). "what" part of the object detection can be fine-grained further i.e. "what type", "what shape" and "what material" etc. This results in the shifting of the object detection tasks to the object description paradigm. Describing an object provides additional detail that enables us to understand the characteristics and attributes of the object ("plastic boat" not just boat, "glass bottle" not just bottle). This additional information can implicitly be used to gain insight into unseen objects (e.g. unknown object is "metallic", "has wheels"), which is not possible in traditional object detection. In this paper, we present a new approach to simultaneously detect objects and infer their attributes, we call it Detect and Describe (DaD) framework. DaD is a deep learning-based approach that extends object detection to object attribute prediction as well. We train our model on aPascal train set and evaluate our approach on aPascal test set. We achieve 97.0% in Area Under the Receiver Operating Characteristic Curve (AUC) for object attributes prediction on aPascal test set. We also show qualitative results for object attribute prediction on unseen objects, which demonstrate the effectiveness of our approach for describing unknown objects.

Related papers

• Generative Region-Language Pretraining for Open-Ended Object Detection [55.42484781608621]
  We propose a framework named GenerateU, which can detect dense objects and generate their names in a free-form way. Our framework achieves comparable results to the open-vocabulary object detection method GLIP.
  arXiv  Detail & Related papers  (2024-03-15T10:52:39Z)
• Measuring the Impact of Scene Level Objects on Object Detection: Towards Quantitative Explanations of Detection Decisions [1.6385815610837167]
  This paper presents a new black box explainability method for additional verification of object detection models. By comparing the accuracies of a model on test data with and without certain scene level objects, the contributions of these objects to the model's performance becomes clearer.
  arXiv  Detail & Related papers  (2024-01-19T16:21:55Z)
• SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving [160.57870373052577]
  We propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. We design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection.
  arXiv  Detail & Related papers  (2023-05-11T16:19:44Z)
• DoUnseen: Tuning-Free Class-Adaptive Object Detection of Unseen Objects for Robotic Grasping [1.6317061277457001]
  We develop an object detector that requires no fine-tuning and can add any object as a class just by capturing a few images of the object. We evaluate our class-adaptive object detector on unseen datasets and compare it to a trained Mask R-CNN on those datasets.
  arXiv  Detail & Related papers  (2023-04-06T02:45:39Z)
• Unsupervised Object Localization: Observing the Background to Discover Objects [4.870509580034194]
  In this work, we take a different approach and propose to look for the background instead. This way, the salient objects emerge as a by-product without any strong assumption on what an object should be. We propose FOUND, a simple model made of a single $conv1times1$ with coarse background masks extracted from self-supervised patch-based representations.
  arXiv  Detail & Related papers  (2022-12-15T13:43:11Z)
• Object Detection in Aerial Images with Uncertainty-Aware Graph Network [61.02591506040606]
  We propose a novel uncertainty-aware object detection framework with a structured-graph, where nodes and edges are denoted by objects. We refer to our model as Uncertainty-Aware Graph network for object DETection (UAGDet)
  arXiv  Detail & Related papers  (2022-08-23T07:29:03Z)
• Automatic dataset generation for specific object detection [6.346581421948067]
  We present a method to synthesize object-in-scene images, which can preserve the objects' detailed features without bringing irrelevant information. Our result shows that in the synthesized image, the boundaries of objects blend very well with the background.
  arXiv  Detail & Related papers  (2022-07-16T07:44:33Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)
• Class-agnostic Object Detection [16.97782147401037]
  We propose class-agnostic object detection as a new problem that focuses on detecting objects irrespective of their object-classes. Specifically, the goal is to predict bounding boxes for all objects in an image but not their object-classes. We propose training and evaluation protocols for benchmarking class-agnostic detectors to advance future research in this domain.
  arXiv  Detail & Related papers  (2020-11-28T19:22:38Z)
• Slender Object Detection: Diagnoses and Improvements [74.40792217534]
  In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely textbfslender objects. For a classical object detection method, a drastic drop of $18.9%$ mAP on COCO is observed, if solely evaluated on slender objects.
  arXiv  Detail & Related papers  (2020-11-17T09:39:42Z)
• Black-box Explanation of Object Detectors via Saliency Maps [66.745167677293]
  We propose D-RISE, a method for generating visual explanations for the predictions of object detectors. We show that D-RISE can be easily applied to different object detectors including one-stage detectors such as YOLOv3 and two-stage detectors such as Faster-RCNN.
  arXiv  Detail & Related papers  (2020-06-05T02:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.