Disentangled Pre-training for Human-Object Interaction Detection

• URL: http://arxiv.org/abs/2404.01725v1
• Date: Tue, 2 Apr 2024 08:21:16 GMT
• Title: Disentangled Pre-training for Human-Object Interaction Detection
• Authors: Zhuolong Li, Xingao Li, Changxing Ding, Xiangmin Xu,
• Abstract summary: We propose an efficient disentangled pre-training method for HOI detection (DP-HOI) DP-HOI utilizes object detection and action recognition datasets to pre-train the detection and interaction decoder layers. It significantly enhances the performance of existing HOI detection models on a broad range of rare categories.
• Score: 22.653500926559833
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting human-object interaction (HOI) has long been limited by the amount of supervised data available. Recent approaches address this issue by pre-training according to pseudo-labels, which align object regions with HOI triplets parsed from image captions. However, pseudo-labeling is tricky and noisy, making HOI pre-training a complex process. Therefore, we propose an efficient disentangled pre-training method for HOI detection (DP-HOI) to address this problem. First, DP-HOI utilizes object detection and action recognition datasets to pre-train the detection and interaction decoder layers, respectively. Then, we arrange these decoder layers so that the pre-training architecture is consistent with the downstream HOI detection task. This facilitates efficient knowledge transfer. Specifically, the detection decoder identifies reliable human instances in each action recognition dataset image, generates one corresponding query, and feeds it into the interaction decoder for verb classification. Next, we combine the human instance verb predictions in the same image and impose image-level supervision. The DP-HOI structure can be easily adapted to the HOI detection task, enabling effective model parameter initialization. Therefore, it significantly enhances the performance of existing HOI detection models on a broad range of rare categories. The code and pre-trained weight are available at https://github.com/xingaoli/DP-HOI.

Related papers

• Hierarchical Graph Interaction Transformer with Dynamic Token Clustering for Camouflaged Object Detection [57.883265488038134]
  We propose a hierarchical graph interaction network termed HGINet for camouflaged object detection. The network is capable of discovering imperceptible objects via effective graph interaction among the hierarchical tokenized features. Our experiments demonstrate the superior performance of HGINet compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-27T12:53:25Z)
• UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection [35.2385914946471]
  We propose a one-stage meta-architecture for HOI detection powered by a novel union-level detector. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time 4x14x.
  arXiv  Detail & Related papers  (2023-12-19T23:34:43Z)
• Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memory [64.11870454160614]
  We propose an efficient Adaptive HOI Detector with Concept-guided Memory (ADA-CM) ADA-CM has two operating modes. The first mode makes it tunable without learning new parameters in a training-free paradigm. Our proposed method achieves competitive results with state-of-the-art on the HICO-DET and V-COCO datasets with much less training time.
  arXiv  Detail & Related papers  (2023-09-07T13:10:06Z)
• DOAD: Decoupled One Stage Action Detection Network [77.14883592642782]
  Localizing people and recognizing their actions from videos is a challenging task towards high-level video understanding. Existing methods are mostly two-stage based, with one stage for person bounding box generation and the other stage for action recognition. We present a decoupled one-stage network dubbed DOAD, to improve the efficiency for-temporal action detection.
  arXiv  Detail & Related papers  (2023-04-01T08:06:43Z)
• Label-Free Synthetic Pretraining of Object Detectors [67.17371526567325]
  We propose a new approach, Synthetic optimized layout with Instance Detection (SOLID), to pretrain object detectors with synthetic images. Our "SOLID" approach consists of two main components: (1) generating synthetic images using a collection of unlabelled 3D models with optimized scene arrangement; (2) pretraining an object detector on "instance detection" task. Our approach does not need any semantic labels for pretraining and allows the use of arbitrary, diverse 3D models.
  arXiv  Detail & Related papers  (2022-08-08T16:55:17Z)
• Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detection [11.928724924319138]
  We propose cross-path consistency learning (CPC) to improve HOI detection for transformers. Our experiments demonstrate the effectiveness of our method, and we achieved significant improvement on V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2022-04-11T02:45:00Z)
• Decoupling Object Detection from Human-Object Interaction Recognition [37.133695677465376]
  DEFR is a DEtection-FRee method to recognize Human-Object Interactions (HOI) at image level without using object location or human pose. We propose two findings to boost the performance of the detection-free approach, which significantly outperforms the detection-assisted state of the arts.
  arXiv  Detail & Related papers  (2021-12-13T03:01:49Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)
• HOTR: End-to-End Human-Object Interaction Detection with Transformers [26.664864824357164]
  We present a novel framework, referred to by HOTR, which directly predicts a set of human, object, interaction> triplets from an image. Our proposed algorithm achieves the state-of-the-art performance in two HOI detection benchmarks with an inference time under 1 ms after object detection.
  arXiv  Detail & Related papers  (2021-04-28T10:10:29Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.