Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle

• URL: http://arxiv.org/abs/2009.11195v1
• Date: Wed, 23 Sep 2020 14:56:19 GMT
• Title: Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle
• Authors: Amr Gomaa, Guillermo Reyes, Alexandra Alles, Lydia Rupp and Michael Feld
• Abstract summary: Hand pointing and eye gaze have been extensively investigated in automotive applications for object selection and referencing. Existing outside-the-vehicle referencing methods focus on a static situation, whereas the situation in a moving vehicle is highly dynamic and subject to safety-critical constraints. We investigate the specific characteristics of each modality and the interaction between them when used in the task of referencing outside objects.
• Score: 58.720142291102135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hand pointing and eye gaze have been extensively investigated in automotive applications for object selection and referencing. Despite significant advances, existing outside-the-vehicle referencing methods consider these modalities separately. Moreover, existing multimodal referencing methods focus on a static situation, whereas the situation in a moving vehicle is highly dynamic and subject to safety-critical constraints. In this paper, we investigate the specific characteristics of each modality and the interaction between them when used in the task of referencing outside objects (e.g. buildings) from the vehicle. We furthermore explore person-specific differences in this interaction by analyzing individuals' performance for pointing and gaze patterns, along with their effect on the driving task. Our statistical analysis shows significant differences in individual behaviour based on object's location (i.e. driver's right side vs. left side), object's surroundings, driving mode (i.e. autonomous vs. normal driving) as well as pointing and gaze duration, laying the foundation for a user-adaptive approach.

Related papers

• Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual Drivers [0.0]
  The rapid advancement of the automotive industry has rendered traditional methods of vehicle interaction, such as touch-based and voice command systems, inadequate for a widening range of non-driving related tasks, such as referencing objects outside of the vehicle. We propose textitIcRegress, a novel regression-based incremental learning approach that adapts to changing behavior and the unique characteristics of drivers engaged in the dual task of driving and referencing objects.
  arXiv  Detail & Related papers  (2024-01-29T12:48:56Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding [50.81809690183755]
  Traffic accidents frequently lead to fatal injuries, contributing to over 50 million deaths until 2023. Previous research primarily assessed the importance of individual participants, treating them as independent entities. We introduce Driving scene Relationship self-Understanding transformer (DRUformer) to enhance the important object detection task.
  arXiv  Detail & Related papers  (2023-11-11T07:26:47Z)
• Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces [0.0]
  Hand gestures, head pose, eye gaze, and speech have been investigated in automotive applications for object selection and referencing. I propose a user-centered adaptive multimodal fusion approach for referencing external objects from a moving vehicle.
  arXiv  Detail & Related papers  (2022-11-07T13:31:00Z)
• Audiovisual Affect Assessment and Autonomous Automobiles: Applications [0.0]
  This contribution aims to foresee according challenges and provide potential avenues towards affect modelling in a multimodal "audiovisual plus x" on the road context. From the technical end, this concerns holistic passenger modelling and reliable diarisation of the individuals in a vehicle. In conclusion, automated affect analysis has just matured to the point of applicability in autonomous vehicles in first selected use-cases.
  arXiv  Detail & Related papers  (2022-03-14T20:39:02Z)
• Multimodal Driver Referencing: A Comparison of Pointing to Objects Inside and Outside the Vehicle [0.0]
  We use multiple modalities to achieve natural human-machine interaction for a specific task. By tracking the movements of eye-gaze, head and finger, we design a multimodal fusion architecture. We propose a method to identity whether the driver's referenced object lies inside or outside the vehicle.
  arXiv  Detail & Related papers  (2022-02-15T12:40:13Z)
• Multimodal Fusion Using Deep Learning Applied to Driver's Referencing of Outside-Vehicle Objects [0.0]
  We utilize deep learning for a multimodal fusion network for referencing objects outside the vehicle. We use features from gaze, head pose and finger pointing simultaneously to precisely predict the referenced objects in different car poses.
  arXiv  Detail & Related papers  (2021-07-26T12:37:06Z)
• Detecting 32 Pedestrian Attributes for Autonomous Vehicles [103.87351701138554]
  In this paper, we address the problem of jointly detecting pedestrians and recognizing 32 pedestrian attributes. We introduce a Multi-Task Learning (MTL) model relying on a composite field framework, which achieves both goals in an efficient way. We show competitive detection and attribute recognition results, as well as a more stable MTL training.
  arXiv  Detail & Related papers  (2020-12-04T15:10:12Z)
• Explicit Domain Adaptation with Loosely Coupled Samples [85.9511585604837]
  We propose a transfer learning framework, core of which is learning an explicit mapping between domains. Due to its interpretability, this is beneficial for safety-critical applications, like autonomous driving.
  arXiv  Detail & Related papers  (2020-04-24T21:23:45Z)
• Parsing-based View-aware Embedding Network for Vehicle Re-Identification [138.11983486734576]
  We propose a parsing-based view-aware embedding network (PVEN) to achieve the view-aware feature alignment and enhancement for vehicle ReID. The experiments conducted on three datasets show that our model outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-04-10T13:06:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.