Sekai: A Video Dataset towards World Exploration

• URL: http://arxiv.org/abs/2506.15675v2
• Date: Fri, 20 Jun 2025 09:03:18 GMT
• Title: Sekai: A Video Dataset towards World Exploration
• Authors: Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, Zhaopan Xu, Xinyue Li, Yukang Feng, Jianwen Sun, Zizhen Li, Fanrui Zhang, Jiaxin Ai, Zhixiang Wang, Yuwei Wu, Tong He, Jiangmiao Pang, Yu Qiao, Yunde Jia, Kaipeng Zhang,
• Abstract summary: Sekai (meaning world'' in Japanese) is a high-quality first-person view worldwide video dataset with rich annotations for world exploration.<n>It consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from over 100 countries and regions across 750 cities.
• Score: 53.151247175736636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video generation techniques have made remarkable progress, promising to be the foundation of interactive world exploration. However, existing video generation datasets are not well-suited for world exploration training as they suffer from some limitations: limited locations, short duration, static scenes, and a lack of annotations about exploration and the world. In this paper, we introduce Sekai (meaning ``world'' in Japanese), a high-quality first-person view worldwide video dataset with rich annotations for world exploration. It consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from over 100 countries and regions across 750 cities. We develop an efficient and effective toolbox to collect, pre-process and annotate videos with location, scene, weather, crowd density, captions, and camera trajectories. Experiments demonstrate the quality of the dataset. And, we use a subset to train an interactive video world exploration model, named YUME (meaning ``dream'' in Japanese). We believe Sekai will benefit the area of video generation and world exploration, and motivate valuable applications. The project page is https://lixsp11.github.io/sekai-project/.

Related papers

• Yume: An Interactive World Generation Model [38.818537395166835]
  Yume aims to use images, text, or videos to create an interactive, realistic, and dynamic world.<n>Method creates a dynamic world from an input image and allows exploration of the world using keyboard actions.
  arXiv  Detail & Related papers  (2025-07-23T17:57:09Z)
• GenWorld: Towards Detecting AI-generated Real-world Simulation Videos [79.98542193919957]
  GenWorld is a large-scale, high-quality, and real-world simulation dataset for AI-generated video detection.<n>We propose a model, SpannDetector, to leverage multi-view consistency as a strong criterion for real-world AI-generated video detection.
  arXiv  Detail & Related papers  (2025-06-12T17:59:33Z)
• WorldExplorer: Towards Generating Fully Navigable 3D Scenes [49.21733308718443]
  WorldExplorer builds fully navigable 3D scenes with consistent visual quality across a wide range of viewpoints.<n>We generate multiple videos along short, pre-defined trajectories, that explore the scene in depth.<n>Our novel scene memory conditions each video on the most relevant prior views, while a collision-detection mechanism prevents degenerate results.
  arXiv  Detail & Related papers  (2025-06-02T15:41:31Z)
• From an Image to a Scene: Learning to Imagine the World from a Million 360 Videos [71.22810401256234]
  Three-dimensional (3D) understanding of objects and scenes play a key role in humans' ability to interact with the world.<n>Large scale synthetic and object-centric 3D datasets have shown to be effective in training models that have 3D understanding of objects.<n>We introduce 360-1M, a 360 video dataset, and a process for efficiently finding corresponding frames from diverse viewpoints at scale.
  arXiv  Detail & Related papers  (2024-12-10T18:59:44Z)
• CityGuessr: City-Level Video Geo-Localization on a Global Scale [54.371452373726584]
  We propose a novel problem of worldwide video geolocalization with the objective of hierarchically predicting the correct city, state/province, country, and continent, given a video. No large scale video datasets that have extensive worldwide coverage exist, to train models for solving this problem. We introduce a new dataset, CityGuessr68k comprising of 68,269 videos from 166 cities all over the world.
  arXiv  Detail & Related papers  (2024-11-10T03:20:00Z)
• WonderWorld: Interactive 3D Scene Generation from a Single Image [38.83667648993784]
  We present WonderWorld, a novel framework for interactive 3D scene generation.<n>WonderWorld generates connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU.
  arXiv  Detail & Related papers  (2024-06-13T17:59:10Z)
• Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception? [57.77643186237265]
  We present Multiview Aerial Visual RECognition or MAVREC, a video dataset where we record synchronized scenes from different perspectives. MAVREC consists of around 2.5 hours of industry-standard 2.7K resolution video sequences, more than 0.5 million frames, and 1.1 million annotated bounding boxes. This makes MAVREC the largest ground and aerial-view dataset, and the fourth largest among all drone-based datasets.
  arXiv  Detail & Related papers  (2023-12-07T18:59:14Z)
• FSVVD: A Dataset of Full Scene Volumetric Video [2.9151420469958533]
  In this paper, we focus on the current most widely used data format, point cloud, and for the first time release a full-scene volumetric video dataset. Comprehensive dataset description and analysis are conducted, with potential usage of this dataset.
  arXiv  Detail & Related papers  (2023-03-07T02:31:08Z)
• Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer [66.56167074658697]
  We present a method that builds on 3D-VQGAN and transformers to generate videos with thousands of frames. Our evaluation shows that our model trained on 16-frame video clips can generate diverse, coherent, and high-quality long videos. We also showcase conditional extensions of our approach for generating meaningful long videos by incorporating temporal information with text and audio.
  arXiv  Detail & Related papers  (2022-04-07T17:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.