PIV3CAMS: a multi-camera dataset for multiple computer vision problems and its application to novel view-point synthesis

• URL: http://arxiv.org/abs/2407.18695v1
• Date: Fri, 26 Jul 2024 12:18:29 GMT
• Title: PIV3CAMS: a multi-camera dataset for multiple computer vision problems and its application to novel view-point synthesis
• Authors: Sohyeong Kim, Martin Danelljan, Radu Timofte, Luc Van Gool, Jean-Philippe Thiran,
• Abstract summary: This thesis introduces Paired Image and Video data from three CAMeraS, namely PIV3CAMS. The PIV3CAMS dataset consists of 8385 pairs of images and 82 pairs of videos taken from three different cameras. In addition to the regeneration of a current state-of-the-art algorithm, we investigate several proposed alternative models that integrate depth information geometrically.
• Score: 120.4361056355332
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The modern approaches for computer vision tasks significantly rely on machine learning, which requires a large number of quality images. While there is a plethora of image datasets with a single type of images, there is a lack of datasets collected from multiple cameras. In this thesis, we introduce Paired Image and Video data from three CAMeraS, namely PIV3CAMS, aimed at multiple computer vision tasks. The PIV3CAMS dataset consists of 8385 pairs of images and 82 pairs of videos taken from three different cameras: Canon D5 Mark IV, Huawei P20, and ZED stereo camera. The dataset includes various indoor and outdoor scenes from different locations in Zurich (Switzerland) and Cheonan (South Korea). Some of the computer vision applications that can benefit from the PIV3CAMS dataset are image/video enhancement, view interpolation, image matching, and much more. We provide a careful explanation of the data collection process and detailed analysis of the data. The second part of this thesis studies the usage of depth information in the view synthesizing task. In addition to the regeneration of a current state-of-the-art algorithm, we investigate several proposed alternative models that integrate depth information geometrically. Through extensive experiments, we show that the effect of depth is crucial in small view changes. Finally, we apply our model to the introduced PIV3CAMS dataset to synthesize novel target views as an example application of PIV3CAMS.

Related papers

• 360 in the Wild: Dataset for Depth Prediction and View Synthesis [66.58513725342125]
  We introduce a large scale 360$circ$ videos dataset in the wild. This dataset has been carefully scraped from the Internet and has been captured from various locations worldwide. Each of the 25K images constituting our dataset is provided with its respective camera's pose and depth map.
  arXiv  Detail & Related papers  (2024-06-27T05:26:38Z)
• NPF-200: A Multi-Modal Eye Fixation Dataset and Method for Non-Photorealistic Videos [51.409547544747284]
  NPF-200 is the first large-scale multi-modal dataset of purely non-photorealistic videos with eye fixations. We conduct a series of analyses to gain deeper insights into this task. We propose a universal frequency-aware multi-modal non-photorealistic saliency detection model called NPSNet.
  arXiv  Detail & Related papers  (2023-08-23T14:25:22Z)
• Replay: Multi-modal Multi-view Acted Videos for Casual Holography [76.49914880351167]
  Replay is a collection of multi-view, multi-modal videos of humans interacting socially. Overall, the dataset contains over 4000 minutes of footage and over 7 million timestamped high-resolution frames. The Replay dataset has many potential applications, such as novel-view synthesis, 3D reconstruction, novel-view acoustic synthesis, human body and face analysis, and training generative models.
  arXiv  Detail & Related papers  (2023-07-22T12:24:07Z)
• AutoDecoding Latent 3D Diffusion Models [95.7279510847827]
  We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core. The 3D autodecoder framework embeds properties learned from the target dataset in the latent space. We then identify the appropriate intermediate volumetric latent space, and introduce robust normalization and de-normalization operations.
  arXiv  Detail & Related papers  (2023-07-07T17:59:14Z)
• EVIMO2: An Event Camera Dataset for Motion Segmentation, Optical Flow, Structure from Motion, and Visual Inertial Odometry in Indoor Scenes with Monocular or Stereo Algorithms [10.058432912712396]
  dataset consists of 41 minutes of data from three 640$times$480 event cameras, one 2080$times$1552 classical color camera. The dataset's 173 sequences are arranged into three categories. Some sequences were recorded in low-light conditions where conventional cameras fail.
  arXiv  Detail & Related papers  (2022-05-06T20:09:18Z)
• Multi-View Video-Based 3D Hand Pose Estimation [11.65577683784217]
  We present the Multi-View Video-Based 3D Hand dataset, consisting of multi-view videos of the hand along with ground-truth 3D pose labels. Our dataset includes more than 402,000 synthetic hand images available in 4,560 videos. Next, we implement MuViHandNet, a neural pipeline consisting of image encoders for obtaining visual embeddings of the hand.
  arXiv  Detail & Related papers  (2021-09-24T05:20:41Z)
• Robust 2D/3D Vehicle Parsing in CVIS [54.825777404511605]
  We present a novel approach to robustly detect and perceive vehicles in different camera views as part of a cooperative vehicle-infrastructure system (CVIS) Our formulation is designed for arbitrary camera views and makes no assumptions about intrinsic or extrinsic parameters. In practice, our approach outperforms SOTA methods on 2D detection, instance segmentation, and 6-DoF pose estimation.
  arXiv  Detail & Related papers  (2021-03-11T03:35:05Z)
• OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving [10.3540046389057]
  This work presents a multi-task visual perception network on unrectified fisheye images. It consists of six primary tasks necessary for an autonomous driving system. We demonstrate that the jointly trained model performs better than the respective single task versions.
  arXiv  Detail & Related papers  (2021-02-15T10:46:24Z)
• YCB-M: A Multi-Camera RGB-D Dataset for Object Recognition and 6DoF Pose Estimation [2.9972063833424216]
  We present a dataset of 32 scenes that have been captured by 7 different 3D cameras, totaling 49,294 frames. This allows evaluating the sensitivity of pose estimation algorithms to the specifics of the used camera.
  arXiv  Detail & Related papers  (2020-04-24T11:14:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.