DIRV: Dense Interaction Region Voting for End-to-End Human-Object Interaction Detection

• URL: http://arxiv.org/abs/2010.01005v2
• Date: Tue, 19 Jan 2021 16:48:22 GMT
• Title: DIRV: Dense Interaction Region Voting for End-to-End Human-Object Interaction Detection
• Authors: Hao-Shu Fang, Yichen Xie, Dian Shao, Cewu Lu
• Abstract summary: We propose a novel one-stage HOI detection approach based on a new concept called interaction region for the HOI problem. Unlike previous methods, our approach concentrates on the densely sampled interaction regions across different scales for each human-object pair. In order to compensate for the detection flaws of a single interaction region, we introduce a novel voting strategy.
• Score: 53.40028068801092
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent years, human-object interaction (HOI) detection has achieved impressive advances. However, conventional two-stage methods are usually slow in inference. On the other hand, existing one-stage methods mainly focus on the union regions of interactions, which introduce unnecessary visual information as disturbances to HOI detection. To tackle the problems above, we propose a novel one-stage HOI detection approach DIRV in this paper, based on a new concept called interaction region for the HOI problem. Unlike previous methods, our approach concentrates on the densely sampled interaction regions across different scales for each human-object pair, so as to capture the subtle visual features that is most essential to the interaction. Moreover, in order to compensate for the detection flaws of a single interaction region, we introduce a novel voting strategy that makes full use of those overlapped interaction regions in place of conventional Non-Maximal Suppression (NMS). Extensive experiments on two popular benchmarks: V-COCO and HICO-DET show that our approach outperforms existing state-of-the-arts by a large margin with the highest inference speed and lightest network architecture. We achieved 56.1 mAP on V-COCO without addtional input. Our code is publicly available at: https://github.com/MVIG-SJTU/DIRV

Related papers

• UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection [35.2385914946471]
  We propose a one-stage meta-architecture for HOI detection powered by a novel union-level detector. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time 4x14x.
  arXiv  Detail & Related papers  (2023-12-19T23:34:43Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• Distance Matters in Human-Object Interaction Detection [22.3445174577181]
  We propose a novel two-stage method for better handling distant interactions in HOI detection. One essential component in our method is a novel Far Near Distance Attention module. Besides, we devise a novel Distance-Aware loss function which leads the model to focus more on distant yet rare interactions.
  arXiv  Detail & Related papers  (2022-07-05T08:06:05Z)
• Interactiveness Field in Human-Object Interactions [89.13149887013905]
  We introduce a previously overlooked interactiveness bimodal prior: given an object in an image, after pairing it with the humans, the generated pairs are either mostly non-interactive, or mostly interactive. We propose new energy constraints based on the cardinality and difference in the inherent "interactiveness field" underlying interactive versus non-interactive pairs. Our method can detect more precise pairs and thus significantly boost HOI detection performance.
  arXiv  Detail & Related papers  (2022-04-16T05:09:25Z)
• ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection [102.9428507180728]
  A common problem in the task of human-object interaction (HOI) detection is that numerous HOI classes have only a small number of labeled examples. We observe that there exist natural correlations and anti-correlations among human-object interactions. We present techniques to learn these priors and leverage them for more effective training, especially on rare classes.
  arXiv  Detail & Related papers  (2021-09-09T06:02:50Z)
• Mining the Benefits of Two-stage and One-stage HOI Detection [26.919979955155664]
  Two-stage methods have dominated Human-Object Interaction (HOI) detection for several years. One-stage methods are challenging to make an appropriate trade-off on multi-task learning, i.e., object detection, and interaction classification. We propose a novel one-stage framework with disentangling human-object detection and interaction classification in a cascade manner.
  arXiv  Detail & Related papers  (2021-08-11T07:38:09Z)
• Detecting Human-Object Interactions with Action Co-occurrence Priors [108.31956827512376]
  A common problem in human-object interaction (HOI) detection task is that numerous HOI classes have only a small number of labeled examples. We observe that there exist natural correlations and anti-correlations among human-object interactions. We present techniques to learn these priors and leverage them for more effective training, especially in rare classes.
  arXiv  Detail & Related papers  (2020-07-17T02:47:45Z)
• Learning Human-Object Interaction Detection using Interaction Points [140.0200950601552]
  We propose a novel fully-convolutional approach that directly detects the interactions between human-object pairs. Our network predicts interaction points, which directly localize and classify the inter-action. Experiments are performed on two popular benchmarks: V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2020-03-31T08:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.