Spatiotemporal Deformation Perception for Fisheye Video Rectification

• URL: http://arxiv.org/abs/2302.03934v1
• Date: Wed, 8 Feb 2023 08:17:50 GMT
• Title: Spatiotemporal Deformation Perception for Fisheye Video Rectification
• Authors: Shangrong Yang, Chunyu Lin, Kang Liao, Yao Zhao
• Abstract summary: We propose a temporal weighting scheme to get a plausible global optical flow. We derive the spatial deformation through the flows of fisheye and distorted-free videos. A temporal deformation aggregator is designed to reconstruct the deformation correlation between frames.
• Score: 44.332845280150785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although the distortion correction of fisheye images has been extensively studied, the correction of fisheye videos is still an elusive challenge. For different frames of the fisheye video, the existing image correction methods ignore the correlation of sequences, resulting in temporal jitter in the corrected video. To solve this problem, we propose a temporal weighting scheme to get a plausible global optical flow, which mitigates the jitter effect by progressively reducing the weight of frames. Subsequently, we observe that the inter-frame optical flow of the video is facilitated to perceive the local spatial deformation of the fisheye video. Therefore, we derive the spatial deformation through the flows of fisheye and distorted-free videos, thereby enhancing the local accuracy of the predicted result. However, the independent correction for each frame disrupts the temporal correlation. Due to the property of fisheye video, a distorted moving object may be able to find its distorted-free pattern at another moment. To this end, a temporal deformation aggregator is designed to reconstruct the deformation correlation between frames and provide a reliable global feature. Our method achieves an end-to-end correction and demonstrates superiority in correction quality and stability compared with the SOTA correction methods.

Related papers

• RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation [88.54817424560056]
  We propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification.
  arXiv  Detail & Related papers  (2024-06-27T06:38:56Z)
• Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction [54.00007868515432]
  Existing methods face challenges in estimating the accurate correction field due to the uniform velocity assumption. We propose a geometry-based Quadratic Rolling Shutter (QRS) motion solver, which precisely estimates the high-order correction field of individual pixels. Our method surpasses the state-of-the-art by +4.98, +0.77, and +4.33 of PSNR on Carla-RS, Fastec-RS, and BS-RSC datasets, respectively.
  arXiv  Detail & Related papers  (2023-03-31T15:09:18Z)
• Deep Rotation Correction without Angle Prior [57.76737888499145]
  We propose a new and practical task, named Rotation Correction, to automatically correct the tilt with high content fidelity. This task can be easily integrated into image editing applications, allowing users to correct the rotated images without any manual operations. We leverage a neural network to predict the optical flows that can warp the tilted images to be perceptually horizontal.
  arXiv  Detail & Related papers  (2022-07-07T02:46:27Z)
• Correcting Face Distortion in Wide-Angle Videos [85.88898349347149]
  We present a video warping algorithm to correct these distortions. Our key idea is to apply stereographic projection locally on the facial regions. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths.
  arXiv  Detail & Related papers  (2021-11-18T21:28:17Z)
• Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow [41.465257944454756]
  We propose a feature-level correction scheme for distortion rectification network. We embed a correction layer in skip-connection and leverage the appearance flows in different layers to pre-correct the image features. It effectively reduces the burden of the decoder by separating content reconstruction and structure correction.
  arXiv  Detail & Related papers  (2021-03-30T02:11:38Z)
• Motion-blurred Video Interpolation and Extrapolation [72.3254384191509]
  We present a novel framework for deblurring, interpolating and extrapolating sharp frames from a motion-blurred video in an end-to-end manner. To ensure temporal coherence across predicted frames and address potential temporal ambiguity, we propose a simple, yet effective flow-based rule.
  arXiv  Detail & Related papers  (2021-03-04T12:18:25Z)
• UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models [8.484676769284578]
  We propose a generic scale-aware self-supervised pipeline for estimating depth, euclidean distance, and visual odometry from unrectified monocular videos. The proposed algorithm is evaluated further on the KITTI rectified dataset, and we achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2020-07-13T20:35:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.