EGO-CH: Dataset and Fundamental Tasks for Visitors BehavioralUnderstanding using Egocentric Vision

• URL: http://arxiv.org/abs/2002.00899v1
• Date: Mon, 3 Feb 2020 17:25:23 GMT
• Title: EGO-CH: Dataset and Fundamental Tasks for Visitors BehavioralUnderstanding using Egocentric Vision
• Abstract summary: We propose EGOcentric-Cultural Heritage (EGO-CH), the first dataset of egocentric videos for visitors' behavior understanding in cultural sites. EGO-CH includes more than $27$ hours of video acquired by $70$ subjects, with labels for $26$ environments and over $200$ different Points of Interest. A large subset of the dataset, consisting of $60$ videos, is associated with surveys filled out by real visitors.
• Score: 24.713448452485714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Equipping visitors of a cultural site with a wearable device allows to easily collect information about their preferences which can be exploited to improve the fruition of cultural goods with augmented reality. Moreover, egocentric video can be processed using computer vision and machine learning to enable an automated analysis of visitors' behavior. The inferred information can be used both online to assist the visitor and offline to support the manager of the site. Despite the positive impact such technologies can have in cultural heritage, the topic is currently understudied due to the limited number of public datasets suitable to study the considered problems. To address this issue, in this paper we propose EGOcentric-Cultural Heritage (EGO-CH), the first dataset of egocentric videos for visitors' behavior understanding in cultural sites. The dataset has been collected in two cultural sites and includes more than $27$ hours of video acquired by $70$ subjects, with labels for $26$ environments and over $200$ different Points of Interest. A large subset of the dataset, consisting of $60$ videos, is associated with surveys filled out by real visitors. To encourage research on the topic, we propose $4$ challenging tasks (room-based localization, point of interest/object recognition, object retrieval and survey prediction) useful to understand visitors' behavior and report baseline results on the dataset.

Related papers

• ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios [50.47300468137957]
  ENIGMA-360 is a new ego-exo dataset acquired in a real industrial scenario.<n>The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized.<n>These results highlight the need for new models capable of robust ego-exo understanding in real-world environments.
  arXiv  Detail & Related papers  (2026-03-10T14:47:15Z)
• BIRD: A Museum Open Dataset Combining Behavior Patterns and Identity Types to Better Model Visitors' Experience [0.0]
  We conducted a study aimed at obtaining comprehensive and detailed information about visitors, their visit experience and their feedback.<n>We built an open dataset combining contextual data (demographic data, preferences, visiting habits, motivations, social context) and behavioral data (spatiotemporal trajectories, gaze data)<n>Our analysis made it possible to re-enact visitor identities combining the majority of characteristics found in the literature and to reproduce the Veron and Levasseur profiles.
  arXiv  Detail & Related papers  (2025-12-29T07:44:32Z)
• Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions [110.43343503158306]
  This paper embeds the manual-assisted task into a vision-language-action framework, where the assistant provides services to the instructor following egocentric vision and commands.<n>Under this setting, we accomplish InterVLA, the first large-scale human-object-human interaction dataset with 11.4 hours and 1.2M frames of multimodal data.<n>We establish novel benchmarks on egocentric human motion estimation, interaction synthesis, and interaction prediction with comprehensive analysis.
  arXiv  Detail & Related papers  (2025-08-06T17:46:23Z)
• EgoLife: Towards Egocentric Life Assistant [60.51196061794498]
  We introduce EgoLife, a project to develop an egocentric life assistant that accompanies and enhances personal efficiency through AI-powered wearable glasses. We conduct a comprehensive data collection study where six participants lived together for one week, continuously recording their daily activities using AI glasses for multimodal egocentric video capture, along with synchronized third-person-view video references. This effort resulted in the EgoLife dataset, a comprehensive 300-hour egocentric, interpersonal, multiview, and multimodal daily life dataset with intensive annotation. We introduce EgoLifeQA, a suite of long-context, life-oriented question-answering tasks designed to provide
  arXiv  Detail & Related papers  (2025-03-05T18:54:16Z)
• Understanding Museum Exhibits using Vision-Language Reasoning [52.35301212718003]
  Museums serve as repositories of cultural heritage and historical artifacts from diverse epochs, civilizations, and regions.<n> domain-specific models are essential for interactive query resolution and gaining historical insights.<n>We collect and curate a large-scale dataset of 65M images and 200M question-answer pairs for exhibits from all around the world.
  arXiv  Detail & Related papers  (2024-12-02T10:54:31Z)
• RoboSense: Large-scale Dataset and Benchmark for Egocentric Robot Perception and Navigation in Crowded and Unstructured Environments [62.5830455357187]
  We setup an egocentric multi-sensor data collection platform based on 3 main types of sensors (Camera, LiDAR and Fisheye) A large-scale multimodal dataset is constructed, named RoboSense, to facilitate egocentric robot perception.
  arXiv  Detail & Related papers  (2024-08-28T03:17:40Z)
• Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learning [80.37314291927889]
  We present EMBED, a method designed to transform exocentric video-language data for egocentric video representation learning. Egocentric videos predominantly feature close-up hand-object interactions, whereas exocentric videos offer a broader perspective on human activities. By applying both vision and language style transfer, our framework creates a new egocentric dataset.
  arXiv  Detail & Related papers  (2024-08-07T06:10:45Z)
• Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Camera [2.427410108595295]
  This paper addresses the daily challenges encountered by visually impaired individuals, such as limited access to information, navigation difficulties, and barriers to social interaction. To alleviate these challenges, we introduce a novel visual question answering dataset. It features videos captured using a 360-degree egocentric wearable camera, enabling observation of the entire surroundings.
  arXiv  Detail & Related papers  (2024-05-30T08:02:05Z)
• The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video [0.9644266648330361]
  The Visual Experience dataset consists of 717 sessions, recorded by 58 observers ranging from 6-49 years old. The VEDB's potential applications are vast, including improving video tracking methodologies, assessing image statistics, and refining deep neural networks for scene and activity recognition.
  arXiv  Detail & Related papers  (2024-02-15T10:34:28Z)
• MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain [23.598727613908853]
  We present MECCANO, a dataset of egocentric videos to study humans behavior understanding in industrial-like settings. The multimodality is characterized by the presence of gaze signals, depth maps and RGB videos acquired simultaneously with a custom headset. The dataset has been explicitly labeled for fundamental tasks in the context of human behavior understanding from a first person view.
  arXiv  Detail & Related papers  (2022-09-19T00:52:42Z)
• Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge [133.80567761430584]
  We collect a large-scale dataset called OVIS for video instance segmentation in the occluded scenario. OVIS consists of 296k high-quality instance masks and 901 occluded scenes. All baseline methods encounter a significant performance degradation of about 80% in the heavily occluded object group.
  arXiv  Detail & Related papers  (2021-11-15T17:59:03Z)
• PEEK: A Large Dataset of Learner Engagement with Educational Videos [20.49299110732228]
  We release a large, novel dataset of learners engaging with educational videos in-the-wild. The dataset, named Personalised Educational Engagement with Knowledge Topics PEEK, is the first publicly available dataset of this nature. We believe that granular learner engagement signals in unison with rich content representations will pave the way to building powerful personalization algorithms.
  arXiv  Detail & Related papers  (2021-09-03T11:23:02Z)
• Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development [6.182409582844314]
  We collect a corpus of about 500 computer vision datasets, from which we sampled 114 dataset publications across different vision tasks. We discuss how computer vision datasets authors value efficiency at the expense of care; universality at the expense of contextuality; and model work at the expense of data work. We conclude with suggestions on how to better incorporate silenced values into the dataset creation and curation process.
  arXiv  Detail & Related papers  (2021-08-09T19:07:58Z)
• An Ethical Highlighter for People-Centric Dataset Creation [62.886916477131486]
  We propose an analytical framework to guide ethical evaluation of existing datasets and to serve future dataset creators in avoiding missteps. Our work is informed by a review and analysis of prior works and highlights where such ethical challenges arise.
  arXiv  Detail & Related papers  (2020-11-27T07:18:44Z)
• Placepedia: Comprehensive Place Understanding with Multi-Faceted Annotations [79.80036503792985]
  We contribute Placepedia, a large-scale place dataset with more than 35M photos from 240K unique places. Besides the photos, each place also comes with massive multi-faceted information, e.g. GDP, population, etc. This dataset, with its large amount of data and rich annotations, allows various studies to be conducted.
  arXiv  Detail & Related papers  (2020-07-07T20:17:01Z)
• REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets [64.76453161039973]
  REVISE (REvealing VIsual biaSEs) is a tool that assists in the investigation of a visual dataset. It surfacing potential biases along three dimensions: (1) object-based, (2) person-based, and (3) geography-based.
  arXiv  Detail & Related papers  (2020-04-16T23:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.