FisheyeEX: Polar Outpainting for Extending the FoV of Fisheye Lens

• URL: http://arxiv.org/abs/2206.05844v1
• Date: Sun, 12 Jun 2022 21:38:50 GMT
• Title: FisheyeEX: Polar Outpainting for Extending the FoV of Fisheye Lens
• Authors: Kang Liao, Chunyu Lin, Yunchao Wei, Yao Zhao
• Abstract summary: Fisheye lens gains increasing applications in computational photography and assisted driving because of its wide field of view (FoV) In this paper, we present a FisheyeEX method that extends the FoV of the fisheye lens by outpainting the invalid regions. The results demonstrate that our approach significantly outperforms the state-of-the-art methods, gaining around 27% more content beyond the original fisheye image.
• Score: 84.12722334460022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fisheye lens gains increasing applications in computational photography and assisted driving because of its wide field of view (FoV). However, the fisheye image generally contains invalid black regions induced by its imaging model. In this paper, we present a FisheyeEX method that extends the FoV of the fisheye lens by outpainting the invalid regions, improving the integrity of captured scenes. Compared with the rectangle and undistorted image, there are two challenges for fisheye image outpainting: irregular painting regions and distortion synthesis. Observing the radial symmetry of the fisheye image, we first propose a polar outpainting strategy to extrapolate the coherent semantics from the center to the outside region. Such an outpainting manner considers the distribution pattern of radial distortion and the circle boundary, boosting a more reasonable completion direction. For the distortion synthesis, we propose a spiral distortion-aware perception module, in which the learning path keeps consistent with the distortion prior of the fisheye image. Subsequently, a scene revision module rearranges the generated pixels with the estimated distortion to match the fisheye image, thus extending the FoV. In the experiment, we evaluate the proposed FisheyeEX on three popular outdoor datasets: Cityscapes, BDD100k, and KITTI, and one real-world fisheye image dataset. The results demonstrate that our approach significantly outperforms the state-of-the-art methods, gaining around 27% more content beyond the original fisheye image.

Related papers

• Combining Generative and Geometry Priors for Wide-Angle Portrait Correction [54.448014761978975]
  We propose encapsulating the generative face prior as a guided natural manifold to facilitate the correction of facial regions. A notable central symmetry relationship exists in the non-face background, yet it has not been explored in the correction process. This geometry prior motivates us to introduce a novel constraint to explicitly enforce symmetry throughout the correction process.
  arXiv  Detail & Related papers  (2024-10-13T16:36:52Z)
• RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation [88.54817424560056]
  We propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification.
  arXiv  Detail & Related papers  (2024-06-27T06:38:56Z)
• Location-guided Head Pose Estimation for Fisheye Image [15.22663220816984]
  Camera with a fisheye or ultra-wide lens covers a wide field of view that cannot be modeled by the perspective projection. Fisheye lens distortion in the peripheral region of the image leads to degraded performance of existing head pose estimation models. This paper presents a new approach for head pose estimation that uses the knowledge of head location in the image to reduce the negative effect of fisheye distortion.
  arXiv  Detail & Related papers  (2024-02-28T13:33:43Z)
• PERF: Panoramic Neural Radiance Field from a Single Panorama [109.31072618058043]
  PERF is a novel view synthesis framework that trains a panoramic neural radiance field from a single panorama. We propose a novel collaborative RGBD inpainting method and a progressive inpainting-and-erasing method to lift up a 360-degree 2D scene to a 3D scene. Our PERF can be widely used for real-world applications, such as panorama-to-3D, text-to-3D, and 3D scene stylization applications.
  arXiv  Detail & Related papers  (2023-10-25T17:59:01Z)
• SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning [105.01294305972037]
  We introduce SimFIR, a framework for fisheye image rectification based on self-supervised representation learning. To learn fine-grained distortion representations, we first split a fisheye image into multiple patches and extract their representations with a Vision Transformer. The transfer performance on the downstream rectification task is remarkably boosted, which verifies the effectiveness of the learned representations.
  arXiv  Detail & Related papers  (2023-08-17T15:20:17Z)
• A Stronger Stitching Algorithm for Fisheye Images based on Deblurring and Registration [3.6417475195085602]
  We devise a stronger stitching algorithm for fisheye images by combining the traditional image processing method with deep learning. In the stage of fisheye image correction, we propose the Attention-based Activation Free Network (ANAFNet) to deblur fisheye images corrected by calibration method. In the part of image registration, we propose the ORB-FREAK-GMS (OFG), a comprehensive image matching algorithm, to improve the accuracy of image registration.
  arXiv  Detail & Related papers  (2023-07-22T06:54:16Z)
• Sector Patch Embedding: An Embedding Module Conforming to The Distortion Pattern of Fisheye Image [23.73394258521532]
  We propose a novel patch embedding method called Sector Patch Embedding(SPE), conforming to the distortion pattern of the fisheye image. The classification top-1 accuracy of ViT and PVT is improved by 0.75% and 2.8% with SPE respectively. Our method can be easily adopted to other Transformer-based models.
  arXiv  Detail & Related papers  (2023-03-26T07:20:02Z)
• FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation [33.71849096992972]
  This paper raises the new task of Fisheye Semantic Completion (FSC), where dense texture, structure, and semantics of a fisheye image are inferred even beyond the sensor field-of-view (FoV) We introduce the new FishDreamer which relies on successful ViTs enhanced with a novel Polar-aware Cross Attention module (PCA) to leverage dense context and guide semantically-consistent content generation.
  arXiv  Detail & Related papers  (2023-03-24T07:34:25Z)
• Rendering Natural Camera Bokeh Effect with Deep Learning [95.86933125733673]
  Bokeh is an important artistic effect used to highlight the main object of interest on the photo. Mobile cameras are unable to produce shallow depth-of-field photos due to a very small aperture diameter of their optics. We propose to learn a realistic shallow focus technique directly from the photos produced by DSLR cameras.
  arXiv  Detail & Related papers  (2020-06-10T07:28:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.