Panonut360: A Head and Eye Tracking Dataset for Panoramic Video

• URL: http://arxiv.org/abs/2403.17708v1
• Date: Tue, 26 Mar 2024 13:54:52 GMT
• Title: Panonut360: A Head and Eye Tracking Dataset for Panoramic Video
• Authors: Yutong Xu, Junhao Du, Jiahe Wang, Yuwei Ning, Sihan Zhou Yang Cao,
• Abstract summary: We present a head and eye tracking dataset involving 50 users watching 15 panoramic videos. The dataset provides details on the viewport and gaze attention locations of users. Our analysis reveals a consistent downward offset in gaze fixations relative to the Field of View.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rapid development and widespread application of VR/AR technology, maximizing the quality of immersive panoramic video services that match users' personal preferences and habits has become a long-standing challenge. Understanding the saliency region where users focus, based on data collected with HMDs, can promote multimedia encoding, transmission, and quality assessment. At the same time, large-scale datasets are essential for researchers and developers to explore short/long-term user behavior patterns and train AI models related to panoramic videos. However, existing panoramic video datasets often include low-frequency user head or eye movement data through short-term videos only, lacking sufficient data for analyzing users' Field of View (FoV) and generating video saliency regions. Driven by these practical factors, in this paper, we present a head and eye tracking dataset involving 50 users (25 males and 25 females) watching 15 panoramic videos. The dataset provides details on the viewport and gaze attention locations of users. Besides, we present some statistics samples extracted from the dataset. For example, the deviation between head and eye movements challenges the widely held assumption that gaze attention decreases from the center of the FoV following a Gaussian distribution. Our analysis reveals a consistent downward offset in gaze fixations relative to the FoV in experimental settings involving multiple users and videos. That's why we name the dataset Panonut, a saliency weighting shaped like a donut. Finally, we also provide a script that generates saliency distributions based on given head or eye coordinates and pre-generated saliency distribution map sets of each video from the collected eye tracking data. The dataset is available on website: https://dianvrlab.github.io/Panonut360/.

Related papers

• CinePile: A Long Video Question Answering Dataset and Benchmark [55.30860239555001]
  We present a novel dataset and benchmark, CinePile, specifically designed for authentic long-form video understanding. Our comprehensive dataset comprises 305,000 multiple-choice questions (MCQs), covering various visual and multimodal aspects. We fine-tuned open-source Video-LLMs on the training split and evaluated both open-source and proprietary video-centric LLMs on the test split of our dataset.
  arXiv  Detail & Related papers  (2024-05-14T17:59:02Z)
• 360+x: A Panoptic Multi-modal Scene Understanding Dataset [13.823967656097146]
  360+x is the first database that covers multiple viewpoints with multiple data modalities to mimic how daily information is accessed in the real world. To the best of our knowledge, this is the first database that covers multiple viewpoints with multiple data modalities to mimic how daily information is accessed in the real world.
  arXiv  Detail & Related papers  (2024-04-01T08:34:42Z)
• VEATIC: Video-based Emotion and Affect Tracking in Context Dataset [34.77364955121413]
  We introduce a brand new large dataset, the Video-based Emotion and Affect Tracking in Context dataset (VEATIC) VEATIC has 124 video clips from Hollywood movies, documentaries, and home videos with continuous valence and arousal ratings of each frame via real-time annotation. Along with the dataset, we propose a new computer vision task to infer the affect of the selected character via both context and character information in each video frame.
  arXiv  Detail & Related papers  (2023-09-13T06:31:35Z)
• NPF-200: A Multi-Modal Eye Fixation Dataset and Method for Non-Photorealistic Videos [51.409547544747284]
  NPF-200 is the first large-scale multi-modal dataset of purely non-photorealistic videos with eye fixations. We conduct a series of analyses to gain deeper insights into this task. We propose a universal frequency-aware multi-modal non-photorealistic saliency detection model called NPSNet.
  arXiv  Detail & Related papers  (2023-08-23T14:25:22Z)
• PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking [90.29143475328506]
  We introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. We animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos.
  arXiv  Detail & Related papers  (2023-07-27T17:58:11Z)
• WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning [70.15653649348674]
  This paper introduces a dynamic blurring (WinDB) fixation collection approach for panoptic video. We have released a new PanopticVideo-300 dataset, containing 300 panoptic clips covering over 225 categories. Using our WinDB approach, there exists frequent and intensive "fixation shifting" - a very special phenomenon that has long been overlooked.
  arXiv  Detail & Related papers  (2023-05-23T10:25:22Z)
• FSVVD: A Dataset of Full Scene Volumetric Video [2.9151420469958533]
  In this paper, we focus on the current most widely used data format, point cloud, and for the first time release a full-scene volumetric video dataset. Comprehensive dataset description and analysis are conducted, with potential usage of this dataset.
  arXiv  Detail & Related papers  (2023-03-07T02:31:08Z)
• HighlightMe: Detecting Highlights from Human-Centric Videos [62.265410865423]
  We present a domain- and user-preference-agnostic approach to detect highlightable excerpts from human-centric videos. We use an autoencoder network equipped with spatial-temporal graph convolutions to detect human activities and interactions. We observe a 4-12% improvement in the mean average precision of matching the human-annotated highlights over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-05T01:18:15Z)
• Video Crowd Localization with Multi-focus Gaussian Neighbor Attention and a Large-Scale Benchmark [35.607604087583425]
  We develop a unified neural network called GNANet to accurately locate head centers in video clips. To facilitate future researches in this field, we introduce a large-scale crowded video benchmark named SenseCrowd. The proposed method is capable to achieve state-of-the-art performance for both video crowd localization and counting.
  arXiv  Detail & Related papers  (2021-07-19T06:59:27Z)
• Towards End-to-end Video-based Eye-Tracking [50.0630362419371]
  Estimating eye-gaze from images alone is a challenging task due to un-observable person-specific factors. We propose a novel dataset and accompanying method which aims to explicitly learn these semantic and temporal relationships. We demonstrate that the fusion of information from visual stimuli as well as eye images can lead towards achieving performance similar to literature-reported figures.
  arXiv  Detail & Related papers  (2020-07-26T12:39:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.