Multi-person eye tracking for real-world scene perception in social settings

• URL: http://arxiv.org/abs/2407.06345v1
• Date: Mon, 8 Jul 2024 19:33:17 GMT
• Title: Multi-person eye tracking for real-world scene perception in social settings
• Authors: Shreshth Saxena, Areez Visram, Neil Lobo, Zahid Mirza, Mehak Rafi Khan, Biranugan Pirabaharan, Alexander Nguyen, Lauren K. Fink,
• Abstract summary: We apply mobile eye tracking in a real-world multi-person setup and develop a system to stream, record, and analyse synchronised data. Our system achieves precise time synchronisation and accurate gaze projection in challenging dynamic scenes. This advancement enables insight into collaborative behaviour, group dynamics, and social interaction, with high ecological validity.
• Score: 34.82692226532414
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Eye movements provide a window into human behaviour, attention, and interaction dynamics. Previous research suggests that eye movements are highly influenced by task, setting, and social others; however, most eye tracking research is conducted in single-person, in-lab settings and is yet to be validated in multi-person, naturalistic contexts. One such prevalent real-world context is the collective viewing of a shared scene in social settings, for example, viewing a concert, film, lecture, sports, etc. Here, we apply mobile eye tracking in a real-world multi-person setup and develop a system to stream, record, and analyse synchronised data. We tested our proposed, open-source system while participants (N=60) watched a live concert and a documentary film screening during a public event. We tackled challenges related to networking bandwidth requirements, real-time monitoring, and gaze projection from individual egocentric perspectives to a common coordinate space for shared gaze analysis. Our system achieves precise time synchronisation and accurate gaze projection in challenging dynamic scenes. Further, to illustrate the potential of collective eye-tracking data, we introduce and evaluate novel analysis metrics and visualisations. Overall, our approach contributes to the development and application of versatile multi-person eye tracking systems in real-world social settings. This advancement enables insight into collaborative behaviour, group dynamics, and social interaction, with high ecological validity. Moreover, it paves the path for innovative, interactive tools that promote collaboration and coordination in social contexts.

Related papers

• Visual-Geometric Collaborative Guidance for Affordance Learning [63.038406948791454]
  We propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues. Our method outperforms the representative models regarding objective metrics and visual quality.
  arXiv  Detail & Related papers  (2024-10-15T07:35:51Z)
• Spherical World-Locking for Audio-Visual Localization in Egocentric Videos [53.658928180166534]
  We propose Spherical World-Locking as a general framework for egocentric scene representation. Compared to conventional head-locked egocentric representations with a 2D planar field-of-view, SWL effectively offsets challenges posed by self-motion. We design a unified encoder-decoder transformer architecture that preserves the spherical structure of the scene representation.
  arXiv  Detail & Related papers  (2024-08-09T22:29:04Z)
• 3D Gaze Tracking for Studying Collaborative Interactions in Mixed-Reality Environments [3.8075244788223044]
  This study presents a novel framework for 3D gaze tracking tailored for mixed-reality settings. Our proposed framework leverages state-of-the-art computer vision and machine learning techniques to overcome obstacles.
  arXiv  Detail & Related papers  (2024-06-16T16:30:56Z)
• I-MPN: Inductive Message Passing Network for Efficient Human-in-the-Loop Annotation of Mobile Eye Tracking Data [4.487146086221174]
  We present a novel human-centered learning algorithm designed for automated object recognition within mobile eye-tracking settings. Our approach seamlessly integrates an object detector with a spatial relation-aware inductive message-passing network (I-MPN), harnessing node profile information and capturing object correlations.
  arXiv  Detail & Related papers  (2024-06-10T13:08:31Z)
• HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understanding [8.10024991952397]
  Existing methods focus on complex interactivities while leveraging a simple relationship model. We propose a new approach named Hierarchical Interlacement Graph (HIG), which leverages a unified layer and graph within a hierarchical structure. Our approach demonstrates superior performance to other methods through extensive experiments conducted in various scenarios.
  arXiv  Detail & Related papers  (2023-12-05T18:47:19Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Do Pedestrians Pay Attention? Eye Contact Detection in the Wild [75.54077277681353]
  In urban environments, humans rely on eye contact for fast and efficient communication with nearby people. In this paper, we focus on eye contact detection in the wild, i.e., real-world scenarios for autonomous vehicles with no control over the environment or the distance of pedestrians. We introduce a model that leverages semantic keypoints to detect eye contact and show that this high-level representation achieves state-of-the-art results on the publicly-available dataset JAAD. To study domain adaptation, we create LOOK: a large-scale dataset for eye contact detection in the wild, which focuses on diverse and un
  arXiv  Detail & Related papers  (2021-12-08T10:21:28Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Non-contact Real time Eye Gaze Mapping System Based on Deep Convolutional Neural Network [0.0]
  We propose a non-contact gaze mapping system applicable in real-world environments. We introduce the GIST Gaze Mapping dataset, a Gaze mapping dataset created to learn and evaluate gaze mapping.
  arXiv  Detail & Related papers  (2020-09-10T02:37:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.