HOTR: End-to-End Human-Object Interaction Detection with Transformers

• URL: http://arxiv.org/abs/2104.13682v1
• Date: Wed, 28 Apr 2021 10:10:29 GMT
• Title: HOTR: End-to-End Human-Object Interaction Detection with Transformers
• Authors: Bumsoo Kim, Junhyun Lee, Jaewoo Kang, Eun-Sol Kim, Hyunwoo J. Kim
• Abstract summary: We present a novel framework, referred to by HOTR, which directly predicts a set of human, object, interaction> triplets from an image. Our proposed algorithm achieves the state-of-the-art performance in two HOI detection benchmarks with an inference time under 1 ms after object detection.
• Score: 26.664864824357164
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-Object Interaction (HOI) detection is a task of identifying "a set of interactions" in an image, which involves the i) localization of the subject (i.e., humans) and target (i.e., objects) of interaction, and ii) the classification of the interaction labels. Most existing methods have indirectly addressed this task by detecting human and object instances and individually inferring every pair of the detected instances. In this paper, we present a novel framework, referred to by HOTR, which directly predicts a set of <human, object, interaction> triplets from an image based on a transformer encoder-decoder architecture. Through the set prediction, our method effectively exploits the inherent semantic relationships in an image and does not require time-consuming post-processing which is the main bottleneck of existing methods. Our proposed algorithm achieves the state-of-the-art performance in two HOI detection benchmarks with an inference time under 1 ms after object detection.

Related papers

• Disentangled Pre-training for Human-Object Interaction Detection [22.653500926559833]
  We propose an efficient disentangled pre-training method for HOI detection (DP-HOI) DP-HOI utilizes object detection and action recognition datasets to pre-train the detection and interaction decoder layers. It significantly enhances the performance of existing HOI detection models on a broad range of rare categories.
  arXiv  Detail & Related papers  (2024-04-02T08:21:16Z)
• UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection [35.2385914946471]
  We propose a one-stage meta-architecture for HOI detection powered by a novel union-level detector. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time 4x14x.
  arXiv  Detail & Related papers  (2023-12-19T23:34:43Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• HODN: Disentangling Human-Object Feature for HOI Detection [51.48164941412871]
  We propose a Human and Object Disentangling Network (HODN) to model the Human-Object Interaction (HOI) relationships explicitly. Considering that human features are more contributive to interaction, we propose a Human-Guide Linking method to make sure the interaction decoder focuses on the human-centric regions. Our proposed method achieves competitive performance on both the V-COCO and the HICO-Det Linking datasets.
  arXiv  Detail & Related papers  (2023-08-20T04:12:50Z)
• Human-Object Interaction Detection via Disentangled Transformer [63.46358684341105]
  We present Disentangled Transformer, where both encoder and decoder are disentangled to facilitate learning of two sub-tasks. Our method outperforms prior work on two public HOI benchmarks by a sizeable margin.
  arXiv  Detail & Related papers  (2022-04-20T08:15:04Z)
• Learning Human-Object Interaction Detection using Interaction Points [140.0200950601552]
  We propose a novel fully-convolutional approach that directly detects the interactions between human-object pairs. Our network predicts interaction points, which directly localize and classify the inter-action. Experiments are performed on two popular benchmarks: V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2020-03-31T08:42:06Z)
• GID-Net: Detecting Human-Object Interaction with Global and Instance Dependency [67.95192190179975]
  We introduce a two-stage trainable reasoning mechanism, referred to as GID block. GID-Net is a human-object interaction detection framework consisting of a human branch, an object branch and an interaction branch. We have compared our proposed GID-Net with existing state-of-the-art methods on two public benchmarks, including V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2020-03-11T11:58:43Z)
• PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection [85.75935399090379]
  We propose a single-stage Human-Object Interaction (HOI) detection method that has outperformed all existing methods on HICO-DET dataset at 37 fps. It is the first real-time HOI detection method.
  arXiv  Detail & Related papers  (2019-12-30T12:00:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.