Domain Adaptation for Large-Vocabulary Object Detectors

• URL: http://arxiv.org/abs/2401.06969v2
• Date: Fri, 10 May 2024 07:15:02 GMT
• Title: Domain Adaptation for Large-Vocabulary Object Detectors
• Authors: Kai Jiang, Jiaxing Huang, Weiying Xie, Jie Lei, Yunsong Li, Ling Shao, Shijian Lu,
• Abstract summary: This paper presents KGD, a Knowledge Graph Distillation technique that exploits the implicit knowledge graphs (KG) in CLIP for effectively adapting LVDs to various downstream domains. Experiments over multiple widely adopted detection benchmarks show that KGD outperforms the state-of-the-art consistently by large margins.
• Score: 103.16365373806829
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-vocabulary object detectors (LVDs) aim to detect objects of many categories, which learn super objectness features and can locate objects accurately while applied to various downstream data. However, LVDs often struggle in recognizing the located objects due to domain discrepancy in data distribution and object vocabulary. At the other end, recent vision-language foundation models such as CLIP demonstrate superior open-vocabulary recognition capability. This paper presents KGD, a Knowledge Graph Distillation technique that exploits the implicit knowledge graphs (KG) in CLIP for effectively adapting LVDs to various downstream domains. KGD consists of two consecutive stages: 1) KG extraction that employs CLIP to encode downstream domain data as nodes and their feature distances as edges, constructing KG that inherits the rich semantic relations in CLIP explicitly; and 2) KG encapsulation that transfers the extracted KG into LVDs to enable accurate cross-domain object classification. In addition, KGD can extract both visual and textual KG independently, providing complementary vision and language knowledge for object localization and object classification in detection tasks over various downstream domains. Experiments over multiple widely adopted detection benchmarks show that KGD outperforms the state-of-the-art consistently by large margins.

Related papers

• GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion [52.026016846945424]
  We propose a new method called GLTW, which encodes the structural information of KGs and merges it with Large Language Models. Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information. Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.
  arXiv  Detail & Related papers  (2025-02-17T06:02:59Z)
• ADKGD: Anomaly Detection in Knowledge Graphs with Dual-Channel Training [38.3788247358102]
  This paper presents an anomaly detection algorithm in knowledge graphs with dual-channel learning (ADKGD) We introduce a kullback-leibler (KL)-loss component to improve the accuracy of the scoring function between the dual channels. Experimental results demonstrate that ADKGD outperforms the state-of-the-art anomaly detection algorithms.
  arXiv  Detail & Related papers  (2025-01-13T06:22:52Z)
• DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection [111.68263493302499]
  We introduce DetCLIPv3, a high-performing detector that excels at both open-vocabulary object detection and hierarchical labels. DetCLIPv3 is characterized by three core designs: 1) Versatile model architecture; 2) High information density data; and 3) Efficient training strategy. DetCLIPv3 demonstrates superior open-vocabulary detection performance, outperforming GLIPv2, GroundingDINO, and DetCLIPv2 by 18.0/19.6/6.6 AP, respectively.
  arXiv  Detail & Related papers  (2024-04-14T11:01:44Z)
• N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fields [112.02885337510716]
  Nested Neural Feature Fields (N2F2) is a novel approach that employs hierarchical supervision to learn a single feature field. We leverage a 2D class-agnostic segmentation model to provide semantically meaningful pixel groupings at arbitrary scales in the image space. Our approach outperforms the state-of-the-art feature field distillation methods on tasks such as open-vocabulary 3D segmentation and localization.
  arXiv  Detail & Related papers  (2024-03-16T18:50:44Z)
• Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learning [13.667326007851674]
  We propose CastDet, a CLIP-activated student-teacher open-vocabulary object detection framework. Our approach boosts not only novel object proposals but also classification. Experimental results demonstrate our CastDet achieving superior open-vocabulary detection performance.
  arXiv  Detail & Related papers  (2023-11-20T10:26:04Z)
• Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural Language Understanding [46.00400830499326]
  We propose a Knowledge-enhanced lANGuAge Representation learning framework for various clOsed dOmains (KANGAROO) In the experiments, we evaluate KANGAROO over various knowledge-aware and general NLP tasks in both full and few-shot learning settings.
  arXiv  Detail & Related papers  (2023-11-12T07:37:24Z)
• HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Models [30.279621764192843]
  Human-Object Interaction (HOI) detection aims to localize human-object pairs and recognize their interactions. Contrastive Language-Image Pre-training (CLIP) has shown great potential in providing interaction prior for HOI detectors. We propose a novel HOI detection framework that efficiently extracts prior knowledge from CLIP and achieves better generalization.
  arXiv  Detail & Related papers  (2023-03-28T07:54:54Z)
• The Overlooked Classifier in Human-Object Interaction Recognition [82.20671129356037]
  We encode the semantic correlation among classes into the classification head by initializing the weights with language embeddings of HOIs. We propose a new loss named LSE-Sign to enhance multi-label learning on a long-tailed dataset. Our simple yet effective method enables detection-free HOI classification, outperforming the state-of-the-arts that require object detection and human pose by a clear margin.
  arXiv  Detail & Related papers  (2022-03-10T23:35:00Z)
• BiDet: An Efficient Binarized Object Detector [96.19708396510894]
  We propose a binarized neural network learning method called BiDet for efficient object detection. Our BiDet fully utilizes the representational capacity of the binary neural networks for object detection by redundancy removal. Our method outperforms the state-of-the-art binary neural networks by a sizable margin.
  arXiv  Detail & Related papers  (2020-03-09T08:16:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.