NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction

• URL: http://arxiv.org/abs/2311.04505v1
• Date: Wed, 8 Nov 2023 07:42:31 GMT
• Title: NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction
• Authors: Thorsten Hempel, Magnus Jung, Ahmed A. Abdelrahman, Ayoub Al-Hamadi
• Abstract summary: We present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions. We make our dataset publicly available to foster further exploration in the field of ego-vision interaction.
• Score: 2.594420805049218
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Eye contact is a crucial non-verbal interaction modality and plays an important role in our everyday social life. While humans are very sensitive to eye contact, the capabilities of machines to capture a person's gaze are still mediocre. We tackle this challenge and present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions, making it a valuable resource for advancing ego-vision-based eye contact research. Our extensive evaluations on NITEC demonstrate strong cross-dataset performance, emphasizing its effectiveness and adaptability in various scenarios, that allows seamless utilization to the fields of computer vision, human-computer interaction, and social robotics. We make our NITEC dataset publicly available to foster reproducibility and further exploration in the field of ego-vision interaction. https://github.com/thohemp/nitec

Related papers

• Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models [5.541130887628606]
  Social dynamics in close human interactions pose significant challenges for Human Mesh Estimation (HME) We introduce a novel data generation method that utilizes Large Vision Language Models (LVLMs) to annotate contact maps which guide test-time optimization to produce paired image and pseudo-ground truth meshes. This methodology not only alleviates the annotation burden but also enables the assembly of a comprehensive dataset specifically tailored for close interactions in HME.
  arXiv  Detail & Related papers  (2024-10-01T01:14:24Z)
• EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision [69.1005706608681]
  We introduce EgoPressure, a novel dataset of touch contact and pressure interaction from an egocentric perspective. EgoPressure comprises 5.0 hours of touch contact and pressure interaction from 21 participants captured by a moving egocentric camera and 7 stationary Kinect cameras.
  arXiv  Detail & Related papers  (2024-09-03T18:53:32Z)
• Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects [89.95728475983263]
  holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. We design the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks.
  arXiv  Detail & Related papers  (2024-03-25T05:12:21Z)
• CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild [18.79132232751083]
  Real-time analysis of the dynamics of the eye region allows us to monitor humans' visual attention allocation and estimate their mental state. We propose CLERA, which achieves precise keypoint detection andtemporal tracking in a joint-learning framework. We also introduce a large-scale dataset of 30k human faces with joint pupil, eye-openness, and landmark annotation.
  arXiv  Detail & Related papers  (2023-06-26T21:20:23Z)
• Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers [0.0]
  We propose a framework that employs Swin Vision Transformers (SwinT) and squeeze and excitation block (SE) to address vision tasks. Our focus was to create an efficient FER model based on SwinT architecture that can recognize facial emotions using minimal data. We trained our model on a hybrid dataset and evaluated its performance on the AffectNet dataset, achieving an F1-score of 0.5420.
  arXiv  Detail & Related papers  (2023-01-26T02:29:17Z)
• A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications [56.458448869572294]
  We introduce DETRtime, a novel framework for time-series segmentation of EEG data. Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront. Our model generalizes well in the task of EEG sleep stage segmentation.
  arXiv  Detail & Related papers  (2022-06-17T10:17:24Z)
• Do Pedestrians Pay Attention? Eye Contact Detection in the Wild [75.54077277681353]
  In urban environments, humans rely on eye contact for fast and efficient communication with nearby people. In this paper, we focus on eye contact detection in the wild, i.e., real-world scenarios for autonomous vehicles with no control over the environment or the distance of pedestrians. We introduce a model that leverages semantic keypoints to detect eye contact and show that this high-level representation achieves state-of-the-art results on the publicly-available dataset JAAD. To study domain adaptation, we create LOOK: a large-scale dataset for eye contact detection in the wild, which focuses on diverse and un
  arXiv  Detail & Related papers  (2021-12-08T10:21:28Z)
• MutualEyeContact: A conversation analysis tool with focus on eye contact [69.17395873398196]
  MutualEyeContact can help scientists to understand the importance of (mutual) eye contact in social interactions. We combine state-of-the-art eye tracking with face recognition based on machine learning and provide a tool for analysis and visualization of social interaction sessions.
  arXiv  Detail & Related papers  (2021-07-09T15:05:53Z)
• A Computer Vision System to Help Prevent the Transmission of COVID-19 [79.62140902232628]
  The COVID-19 pandemic affects every area of daily life globally. Health organizations advise social distancing, wearing face mask, and avoiding touching face. We developed a deep learning-based computer vision system to help prevent the transmission of COVID-19.
  arXiv  Detail & Related papers  (2021-03-16T00:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.