Contrastive Object Detection Using Knowledge Graph Embeddings

• URL: http://arxiv.org/abs/2112.11366v1
• Date: Tue, 21 Dec 2021 17:10:21 GMT
• Title: Contrastive Object Detection Using Knowledge Graph Embeddings
• Authors: Christopher Lang, Alexander Braun, Abhinav Valada
• Abstract summary: We compare the error statistics of the class embeddings learned from a one-hot approach with semantically structured embeddings from natural language processing or knowledge graphs. We propose a knowledge-embedded design for keypoint-based and transformer-based object detection architectures.
• Score: 72.17159795485915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object recognition for the most part has been approached as a one-hot problem that treats classes to be discrete and unrelated. Each image region has to be assigned to one member of a set of objects, including a background class, disregarding any similarities in the object types. In this work, we compare the error statistics of the class embeddings learned from a one-hot approach with semantically structured embeddings from natural language processing or knowledge graphs that are widely applied in open world object detection. Extensive experimental results on multiple knowledge-embeddings as well as distance metrics indicate that knowledge-based class representations result in more semantically grounded misclassifications while performing on par compared to one-hot methods on the challenging COCO and Cityscapes object detection benchmarks. We generalize our findings to multiple object detection architectures by proposing a knowledge-embedded design for keypoint-based and transformer-based object detection architectures.

Related papers

• CoTDet: Affordance Knowledge Prompting for Task Driven Object Detection [42.2847114428716]
  Task driven object detection aims to detect object instances suitable for affording a task in an image. Its challenge lies in object categories available for the task being too diverse to be limited to a closed set of object vocabulary for traditional object detection. We propose to explore fundamental affordances rather than object categories, i.e., common attributes that enable different objects to accomplish the same task.
  arXiv  Detail & Related papers  (2023-09-03T06:18:39Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Robust Region Feature Synthesizer for Zero-Shot Object Detection [87.79902339984142]
  We build a novel zero-shot object detection framework that contains an Intra-class Semantic Diverging component and an Inter-class Structure Preserving component. It is the first study to carry out zero-shot object detection in remote sensing imagery.
  arXiv  Detail & Related papers  (2022-01-01T03:09:15Z)
• Objects in Semantic Topology [36.297624587122506]
  A qualified open-world object detector can not only identify objects of known categories, but also discover unknown objects. We provide a unified perspective: Semantic Topology. Experiments demonstrate that semantic topology, either randomly-generated or derived from a well-trained language model, could outperform the current state-of-the-art open-world object detectors.
  arXiv  Detail & Related papers  (2021-10-06T12:15:30Z)
• A Survey of Modern Deep Learning based Object Detection Models [0.7388859384645263]
  This article surveys recent developments in deep learning based object detectors. It provides an overview of benchmark datasets and evaluation metrics used in detection. It also covers contemporary lightweight classification models used on edge devices.
  arXiv  Detail & Related papers  (2021-04-24T06:33:54Z)
• Class-agnostic Object Detection [16.97782147401037]
  We propose class-agnostic object detection as a new problem that focuses on detecting objects irrespective of their object-classes. Specifically, the goal is to predict bounding boxes for all objects in an image but not their object-classes. We propose training and evaluation protocols for benchmarking class-agnostic detectors to advance future research in this domain.
  arXiv  Detail & Related papers  (2020-11-28T19:22:38Z)
• Slender Object Detection: Diagnoses and Improvements [74.40792217534]
  In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely textbfslender objects. For a classical object detection method, a drastic drop of $18.9%$ mAP on COCO is observed, if solely evaluated on slender objects.
  arXiv  Detail & Related papers  (2020-11-17T09:39:42Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)
• Look-into-Object: Self-supervised Structure Modeling for Object Recognition [71.68524003173219]
  We propose to "look into object" (explicitly yet intrinsically model the object structure) through incorporating self-supervisions. We show the recognition backbone can be substantially enhanced for more robust representation learning. Our approach achieves large performance gain on a number of benchmarks, including generic object recognition (ImageNet) and fine-grained object recognition tasks (CUB, Cars, Aircraft)
  arXiv  Detail & Related papers  (2020-03-31T12:22:51Z)
• Evaluating Salient Object Detection in Natural Images with Multiple Objects having Multi-level Saliency [3.464871689508835]
  Salient object detection is evaluated using binary ground truth with the labels being salient object class and background. Our dataset, named SalMoN (saliency in multi-object natural images), has 588 images containing multiple objects.
  arXiv  Detail & Related papers  (2020-03-19T00:06:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.