The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video

• URL: http://arxiv.org/abs/2404.18934v2
• Date: Tue, 13 Aug 2024 16:01:14 GMT
• Title: The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video
• Abstract summary: The Visual Experience dataset consists of 717 sessions, recorded by 58 observers ranging from 6-49 years old. The VEDB's potential applications are vast, including improving video tracking methodologies, assessing image statistics, and refining deep neural networks for scene and activity recognition.
• Score: 0.9644266648330361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce the Visual Experience Dataset (VEDB), a compilation of over 240 hours of egocentric video combined with gaze- and head-tracking data that offers an unprecedented view of the visual world as experienced by human observers. The dataset consists of 717 sessions, recorded by 58 observers ranging from 6-49 years old. This paper outlines the data collection, processing, and labeling protocols undertaken to ensure a representative sample and discusses the potential sources of error or bias within the dataset. The VEDB's potential applications are vast, including improving gaze tracking methodologies, assessing spatiotemporal image statistics, and refining deep neural networks for scene and activity recognition. The VEDB is accessible through established open science platforms and is intended to be a living dataset with plans for expansion and community contributions. It is released with an emphasis on ethical considerations, such as participant privacy and the mitigation of potential biases. By providing a dataset grounded in real-world experiences and accompanied by extensive metadata and supporting code, the authors invite the research community to utilize and contribute to the VEDB, facilitating a richer understanding of visual perception and behavior in naturalistic settings.

Related papers

• ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios [50.47300468137957]
  ENIGMA-360 is a new ego-exo dataset acquired in a real industrial scenario.<n>The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized.<n>These results highlight the need for new models capable of robust ego-exo understanding in real-world environments.
  arXiv  Detail & Related papers  (2026-03-10T14:47:15Z)
• Revisiting Salient Object Detection from an Observer-Centric Perspective [48.99721284788945]
  We propose Observer-Centric Salient Object Detection (OC-SOD), where salient regions are predicted by considering not only the visual cues but also the observer-specific factors such as their preferences or intents.<n>As a result, this formulation captures the intrinsic ambiguity and diversity of human perception, enabling personalized and context-aware saliency prediction.
  arXiv  Detail & Related papers  (2026-02-06T03:53:01Z)
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision [72.84868704100595]
  This paper presents a dataset specifically designed for self-supervision on remote sensing data, intended to enhance deep learning applications on Earth monitoring tasks. The dataset spans 15 tera pixels of global remote-sensing data, combining imagery from a diverse range of sources, including NEON, Sentinel, and a novel release of 1m spatial resolution data from Satellogic. Accompanying the dataset is EarthMAE, a tailored Masked Autoencoder developed to tackle the distinct challenges of remote sensing data.
  arXiv  Detail & Related papers  (2025-01-14T13:42:22Z)
• A Dataset for Crucial Object Recognition in Blind and Low-Vision Individuals' Navigation [4.413331329339185]
  This paper introduces a dataset for improving real-time object recognition systems to aid blind and low-vision (BLV) individuals in navigation tasks. The dataset comprises 21 videos of BLV individuals navigating outdoor spaces, and a taxonomy of 90 objects crucial for BLV navigation, refined through a focus group study.
  arXiv  Detail & Related papers  (2024-07-23T18:19:27Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  We present a comprehensive dataset compiled from Nature Communications articles covering 72 scientific fields. We evaluated 19 proprietary and open-source models on two benchmark tasks, figure captioning and multiple-choice, and conducted human expert annotation. Fine-tuning Qwen2-VL-7B with our task-specific data achieved better performance than GPT-4o and even human experts in multiple-choice evaluations.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• Panonut360: A Head and Eye Tracking Dataset for Panoramic Video [0.0]
  We present a head and eye tracking dataset involving 50 users watching 15 panoramic videos. The dataset provides details on the viewport and gaze attention locations of users. Our analysis reveals a consistent downward offset in gaze fixations relative to the Field of View.
  arXiv  Detail & Related papers  (2024-03-26T13:54:52Z)
• Concept Drift and Long-Tailed Distribution in Fine-Grained Visual Categorization: Benchmark and Method [84.68818879525568]
  We present a Concept Drift and Long-Tailed Distribution dataset. The characteristics of instances tend to vary with time and exhibit a long-tailed distribution. We propose a feature recombination framework to address the learning challenges associated with CDLT.
  arXiv  Detail & Related papers  (2023-06-04T12:42:45Z)
• Towards Precision in Appearance-based Gaze Estimation in the Wild [3.4253416336476246]
  We present a large gaze estimation dataset, PARKS-Gaze, with wider head pose and illumination variation. The proposed dataset is more challenging and enable models to generalize on unseen participants better than the existing in-the-wild datasets.
  arXiv  Detail & Related papers  (2023-02-05T10:09:35Z)
• Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines [9.896915478880635]
  The degree of concentration, enthusiasm, optimism, and passion displayed by individual(s) while interacting with a machine is referred to as user engagement' To create engagement prediction systems that can work in real-world conditions, it is quintessential to learn from rich, diverse datasets. Large scale multi-faceted engagement in the wild dataset EngageNet is proposed.
  arXiv  Detail & Related papers  (2023-02-01T13:25:54Z)
• T2FPV: Constructing High-Fidelity First-Person View Datasets From Real-World Pedestrian Trajectories [9.44806128120871]
  We present T2FPV, a method for constructing high-fidelity first-person view datasets given a real-world, top-down trajectory dataset. We showcase our approach on the ETH/UCY pedestrian dataset to generate the egocentric visual data of all interacting pedestrians.
  arXiv  Detail & Related papers  (2022-09-22T20:14:43Z)
• Visual Distant Supervision for Scene Graph Generation [66.10579690929623]
  Scene graph models usually require supervised learning on large quantities of labeled data with intensive human annotation. We propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data. Comprehensive experimental results show that our distantly supervised model outperforms strong weakly supervised and semi-supervised baselines.
  arXiv  Detail & Related papers  (2021-03-29T06:35:24Z)
• Towards End-to-end Video-based Eye-Tracking [50.0630362419371]
  Estimating eye-gaze from images alone is a challenging task due to un-observable person-specific factors. We propose a novel dataset and accompanying method which aims to explicitly learn these semantic and temporal relationships. We demonstrate that the fusion of information from visual stimuli as well as eye images can lead towards achieving performance similar to literature-reported figures.
  arXiv  Detail & Related papers  (2020-07-26T12:39:15Z)
• Open Graph Benchmark: Datasets for Machine Learning on Graphs [86.96887552203479]
  We present the Open Graph Benchmark (OGB) to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics.
  arXiv  Detail & Related papers  (2020-05-02T03:09:50Z)
• REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets [64.76453161039973]
  REVISE (REvealing VIsual biaSEs) is a tool that assists in the investigation of a visual dataset. It surfacing potential biases along three dimensions: (1) object-based, (2) person-based, and (3) geography-based.
  arXiv  Detail & Related papers  (2020-04-16T23:54:37Z)
• GraspNet: A Large-Scale Clustered and Densely Annotated Dataset for Object Grasping [49.777649953381676]
  We contribute a large-scale grasp pose detection dataset with a unified evaluation system. Our dataset contains 87,040 RGBD images with over 370 million grasp poses.
  arXiv  Detail & Related papers  (2019-12-31T18:15:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.