The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain

• URL: http://arxiv.org/abs/2010.05654v1
• Date: Mon, 12 Oct 2020 12:50:30 GMT
• Title: The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain
• Authors: Francesco Ragusa and Antonino Furnari and Salvatore Livatino and Giovanni Maria Farinella
• Abstract summary: We introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings. The dataset has been explicitly labeled for the task of recognizing human-object interactions from an egocentric perspective. Baseline results show that the MECCANO dataset is a challenging benchmark to study egocentric human-object interactions in industrial-like scenarios.
• Score: 20.99718135562034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Wearable cameras allow to collect images and videos of humans interacting with the world. While human-object interactions have been thoroughly investigated in third person vision, the problem has been understudied in egocentric settings and in industrial scenarios. To fill this gap, we introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings. MECCANO has been acquired by 20 participants who were asked to build a motorbike model, for which they had to interact with tiny objects and tools. The dataset has been explicitly labeled for the task of recognizing human-object interactions from an egocentric perspective. Specifically, each interaction has been labeled both temporally (with action segments) and spatially (with active object bounding boxes). With the proposed dataset, we investigate four different tasks including 1) action recognition, 2) active object detection, 3) active object recognition and 4) egocentric human-object interaction detection, which is a revisited version of the standard human-object interaction detection task. Baseline results show that the MECCANO dataset is a challenging benchmark to study egocentric human-object interactions in industrial-like scenarios. We publicy release the dataset at https://iplab.dmi.unict.it/MECCANO.

Related papers

• EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Views [51.53089073920215]
  Understanding egocentric human-object interaction (HOI) is a fundamental aspect of human-centric perception. Existing methods primarily leverage observations of HOI to capture interaction regions from an exocentric view. We present EgoChoir, which links object structures with interaction contexts inherent in appearance and head motion to reveal object affordance.
  arXiv  Detail & Related papers  (2024-05-22T14:03:48Z)
• HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment [43.6454394625555]
  HOI-M3 is a novel large-scale dataset for modeling the interactions of Multiple huMans and Multiple objects. It provides accurate 3D tracking for both humans and objects from dense RGB and object-mounted IMU inputs.
  arXiv  Detail & Related papers  (2024-03-30T09:24:25Z)
• HODN: Disentangling Human-Object Feature for HOI Detection [51.48164941412871]
  We propose a Human and Object Disentangling Network (HODN) to model the Human-Object Interaction (HOI) relationships explicitly. Considering that human features are more contributive to interaction, we propose a Human-Guide Linking method to make sure the interaction decoder focuses on the human-centric regions. Our proposed method achieves competitive performance on both the V-COCO and the HICO-Det Linking datasets.
  arXiv  Detail & Related papers  (2023-08-20T04:12:50Z)
• Full-Body Articulated Human-Object Interaction [61.01135739641217]
  CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions. CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process. By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
  arXiv  Detail & Related papers  (2022-12-20T19:50:54Z)
• MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain [23.598727613908853]
  We present MECCANO, a dataset of egocentric videos to study humans behavior understanding in industrial-like settings. The multimodality is characterized by the presence of gaze signals, depth maps and RGB videos acquired simultaneously with a custom headset. The dataset has been explicitly labeled for fundamental tasks in the context of human behavior understanding from a first person view.
  arXiv  Detail & Related papers  (2022-09-19T00:52:42Z)
• BEHAVE: Dataset and Method for Tracking Human Object Interactions [105.77368488612704]
  We present the first full body human- object interaction dataset with multi-view RGBD frames and corresponding 3D SMPL and object fits along with the annotated contacts between them. We use this data to learn a model that can jointly track humans and objects in natural environments with an easy-to-use portable multi-camera setup.
  arXiv  Detail & Related papers  (2022-04-14T13:21:19Z)
• Detecting Human-to-Human-or-Object (H2O) Interactions with DIABOLO [29.0200561485714]
  We propose a new interaction dataset to deal with both types of human interactions: Human-to-Human-or-Object (H2O) In addition, we introduce a novel taxonomy of verbs, intended to be closer to a description of human body attitude in relation to the surrounding targets of interaction. We propose DIABOLO, an efficient subject-centric single-shot method to detect all interactions in one forward pass.
  arXiv  Detail & Related papers  (2022-01-07T11:00:11Z)
• EgoBody: Human Body Shape, Motion and Social Interactions from Head-Mounted Devices [76.50816193153098]
  EgoBody is a novel large-scale dataset for social interactions in complex 3D scenes. We employ Microsoft HoloLens2 headsets to record rich egocentric data streams including RGB, depth, eye gaze, head and hand tracking. To obtain accurate 3D ground-truth, we calibrate the headset with a multi-Kinect rig and fit expressive SMPL-X body meshes to multi-view RGB-D frames.
  arXiv  Detail & Related papers  (2021-12-14T18:41:28Z)
• Human-Object Interaction Detection:A Quick Survey and Examination of Methods [17.8805983491991]
  This is the first general survey of the state-of-the-art and milestone works in this field. We provide a basic survey of the developments in the field of human-object interaction detection. We examine the HORCNN architecture as it is a foundational work in the field.
  arXiv  Detail & Related papers  (2020-09-27T20:58:39Z)
• Learning Human-Object Interaction Detection using Interaction Points [140.0200950601552]
  We propose a novel fully-convolutional approach that directly detects the interactions between human-object pairs. Our network predicts interaction points, which directly localize and classify the inter-action. Experiments are performed on two popular benchmarks: V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2020-03-31T08:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.