360{\deg} Optical Flow using Tangent Images

• URL: http://arxiv.org/abs/2112.14331v1
• Date: Tue, 28 Dec 2021 23:50:46 GMT
• Title: 360{\deg} Optical Flow using Tangent Images
• Authors: Mingze Yuan, Christian Richardt
• Abstract summary: equirectangular projection (ERP) is the most common format for storing, processing and visualising 360deg images. We propose a 360deg optical flow method based on tangent images.
• Score: 18.146747748702513
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Omnidirectional 360{\deg} images have found many promising and exciting applications in computer vision, robotics and other fields, thanks to their increasing affordability, portability and their 360{\deg} field of view. The most common format for storing, processing and visualising 360{\deg} images is equirectangular projection (ERP). However, the distortion introduced by the nonlinear mapping from 360{\deg} image to ERP image is still a barrier that holds back ERP images from being used as easily as conventional perspective images. This is especially relevant when estimating 360{\deg} optical flow, as the distortions need to be mitigated appropriately. In this paper, we propose a 360{\deg} optical flow method based on tangent images. Our method leverages gnomonic projection to locally convert ERP images to perspective images, and uniformly samples the ERP image by projection to a cubemap and regular icosahedron vertices, to incrementally refine the estimated 360{\deg} flow fields even in the presence of large rotations. Our experiments demonstrate the benefits of our proposed method both quantitatively and qualitatively.

Related papers

• ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings [48.72040500647568]
  We present ODGS, a novelization pipeline for omnidirectional images, with geometric interpretation. The entire pipeline is parallelized using, achieving optimization and speeds 100 times faster than NeRF-based methods. Results show ODGS restores fine details effectively, even when reconstructing large 3D scenes.
  arXiv  Detail & Related papers  (2024-10-28T02:45:13Z)
• Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors [51.36238367193988]
  We tackle sparse-view reconstruction of a 360 3D scene using priors from latent diffusion models (LDM) We present SparseSplat360, a method that employs a cascade of in-painting and artifact removal models to fill in missing details and clean novel views. Our method generates entire 360 scenes from as few as 9 input views, with a high degree of foreground and background detail.
  arXiv  Detail & Related papers  (2024-05-26T11:01:39Z)
• Perceptual Quality Assessment of 360$^\circ$ Images Based on Generative Scanpath Representation [40.00063797833765]
  We introduce a unique generative scanpath representation (GSR) for effective quality inference of 360$circ$ images. GSR aggregates varied perceptual experiences of multi-hypothesis users under a predefined viewing condition. We then propose an efficient OIQA computational framework by learning the quality maps GSR.
  arXiv  Detail & Related papers  (2023-09-07T04:10:30Z)
• NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes [59.15910989235392]
  We introduce NeO 360, Neural fields for sparse view synthesis of outdoor scenes. NeO 360 is a generalizable method that reconstructs 360deg scenes from a single or a few posed RGB images. Our representation combines the best of both voxel-based and bird's-eye-view (BEV) representations.
  arXiv  Detail & Related papers  (2023-08-24T17:59:50Z)
• Distortion-Aware Self-Supervised 360{\deg} Depth Estimation from A Single Equirectangular Projection Image [35.943763515381214]
  This paper proposes a new technique for single 360deg image depth prediction under open environments. One is the limitation of supervision datasets - the currently available dataset is limited to indoor scenes. The other is the problems caused by Equirectangular Projection Format (ERP), commonly used for 360deg images, that are coordinate and distortion.
  arXiv  Detail & Related papers  (2022-04-03T08:28:44Z)
• Field-of-View IoU for Object Detection in 360{\deg} Images [36.72543749626039]
  We propose two fundamental techniques -- Field-of-View IoU (FoV-IoU) and 360Augmentation for object detection in 360deg images. FoV-IoU computes the intersection-over-union of two Field-of-View bounding boxes in a spherical image which could be used for training, inference, and evaluation. 360Augmentation is a data augmentation technique specific to 360deg object detection task which randomly rotates a spherical image and solves the bias due to the sphere-to-plane projection.
  arXiv  Detail & Related papers  (2022-02-07T14:01:59Z)
• SphereSR: 360{\deg} Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation [27.10716804733828]
  We propose a novel framework to generate a continuous spherical image representation from an LR 360degimage. Specifically, we first propose a feature extraction module that represents the spherical data based on icosahedron. We then propose a spherical local implicit image function (SLIIF) to predict RGB values at the spherical coordinates.
  arXiv  Detail & Related papers  (2021-12-13T10:16:51Z)
• 360MonoDepth: High-Resolution 360{\deg} Monocular Depth Estimation [15.65828728205071]
  monocular depth estimation remains a challenge for 360deg data. Current CNN-based methods do not support such high resolutions due to limited GPU memory. We propose a flexible framework for monocular depth estimation from high-resolution 360deg images using tangent images.
  arXiv  Detail & Related papers  (2021-11-30T18:57:29Z)
• Extreme Rotation Estimation using Dense Correlation Volumes [73.35119461422153]
  We present a technique for estimating the relative 3D rotation of an RGB image pair in an extreme setting. We observe that, even when images do not overlap, there may be rich hidden cues as to their geometric relationship. We propose a network design that can automatically learn such implicit cues by comparing all pairs of points between the two input images.
  arXiv  Detail & Related papers  (2021-04-28T02:00:04Z)
• Robust 360-8PA: Redesigning The Normalized 8-point Algorithm for 360-FoV Images [53.11097060367591]
  We present a novel strategy for estimating an essential matrix from 360-FoV images in spherical projection. We show that our normalization can increase the camera pose accuracy by about 20% without significantly overhead the time.
  arXiv  Detail & Related papers  (2021-04-22T07:23:11Z)
• Visual Question Answering on 360{\deg} Images [96.00046925811515]
  VQA 360 is a novel task of visual question answering on 360 images. We collect the first VQA 360 dataset, containing around 17,000 real-world image-question-answer triplets for a variety of question types.
  arXiv  Detail & Related papers  (2020-01-10T08:18:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.