A Novel Viewport-Adaptive Motion Compensation Technique for Fisheye Video

• URL: http://arxiv.org/abs/2202.13892v1
• Date: Mon, 28 Feb 2022 15:41:08 GMT
• Title: A Novel Viewport-Adaptive Motion Compensation Technique for Fisheye Video
• Authors: Andy Regensky, Christian Herglotz, Andr\'e Kaup
• Abstract summary: A recently proposed projection-based approach takes the fisheye projection into account to improve fisheye motion compensation. We propose a novel viewport-adaptive motion compensation technique that applies the motion vectors in different perspective viewports. We achieve average gains of +2.40 dB in terms of PSNR compared to the state of the art in fisheye motion compensation.
• Score: 7.09875977818162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although fisheye cameras are in high demand in many application areas due to their large field of view, many image and video signal processing tasks such as motion compensation suffer from the introduced strong radial distortions. A recently proposed projection-based approach takes the fisheye projection into account to improve fisheye motion compensation. However, the approach does not consider the large field of view of fisheye lenses that requires the consideration of different motion planes in 3D space. We propose a novel viewport-adaptive motion compensation technique that applies the motion vectors in different perspective viewports in order to realize these motion planes. Thereby, some pixels are mapped to so-called virtual image planes and require special treatment to obtain reliable mappings between the perspective viewports and the original fisheye image. While the state-of-the-art ultra wide-angle compensation is sufficiently accurate, we propose a virtual image plane compensation that leads to perfect mappings. All in all, we achieve average gains of +2.40 dB in terms of PSNR compared to the state of the art in fisheye motion compensation.

Related papers

• FisheyeGaussianLift: BEV Feature Lifting for Surround-View Fisheye Camera Perception [0.8374040635931297]
  We present a distortion-aware BEV segmentation framework that processes multi-camera high-resolution fisheye images.<n>Each image pixel is lifted into 3D space via Gaussian parameterization, predicting spatial means and anisotropic covariances to explicitly model geometric uncertainty.<n>Experiments demonstrate strong segmentation performance on complex parking and urban driving scenarios, achieving IoU scores of 87.75% for drivable regions and 57.26% for vehicles under severe fisheye distortion.
  arXiv  Detail & Related papers  (2025-11-21T12:42:07Z)
• Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Images [8.719294151596705]
  We evaluate the impact of pinhole, equidistant and double sphere camera models, as well as cylindrical projection methods, on 3D human pose estimation accuracy.<n>We find that in close-up scenarios, pinhole projection is inadequate, and the optimal projection method varies with the FOV covered by the human pose.<n>We propose a for selecting the appropriate projection model based on the detection bounding box to enhance prediction quality.
  arXiv  Detail & Related papers  (2025-06-24T16:05:36Z)
• Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras [17.75361921797657]
  We introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods.
  arXiv  Detail & Related papers  (2024-09-07T07:53:40Z)
• RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation [88.54817424560056]
  We propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification.
  arXiv  Detail & Related papers  (2024-06-27T06:38:56Z)
• Cinematic Behavior Transfer via NeRF-based Differentiable Filming [63.1622492808519]
  Existing SLAM methods face limitations in dynamic scenes and human pose estimation often focuses on 2D projections. We first introduce a reverse filming behavior estimation technique. We then introduce a cinematic transfer pipeline that is able to transfer various shot types to a new 2D video or a 3D virtual environment.
  arXiv  Detail & Related papers  (2023-11-29T15:56:58Z)
• 3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields [58.6780687018956]
  We present a 3D motion magnification method that can magnify subtle motions from scenes captured by a moving camera. We represent the scene with time-varying radiance fields and leverage the Eulerian principle for motion magnification. We evaluate the effectiveness of our method on both synthetic and real-world scenes captured under various camera setups.
  arXiv  Detail & Related papers  (2023-08-07T17:59:59Z)
• Depth Estimation Analysis of Orthogonally Divergent Fisheye Cameras with Distortion Removal [0.0]
  Traditional stereo vision systems may not be suitable for certain scenarios due to their limited field of view. Fisheye cameras introduce significant distortion at the edges that affects the accuracy of stereo matching and depth estimation. This paper proposes a method for distortion-removal and depth estimation analysis for stereovision system.
  arXiv  Detail & Related papers  (2023-07-07T13:44:12Z)
• FisheyeEX: Polar Outpainting for Extending the FoV of Fisheye Lens [84.12722334460022]
  Fisheye lens gains increasing applications in computational photography and assisted driving because of its wide field of view (FoV) In this paper, we present a FisheyeEX method that extends the FoV of the fisheye lens by outpainting the invalid regions. The results demonstrate that our approach significantly outperforms the state-of-the-art methods, gaining around 27% more content beyond the original fisheye image.
  arXiv  Detail & Related papers  (2022-06-12T21:38:50Z)
• Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos [115.71874459429381]
  We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. Experiments on benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.
  arXiv  Detail & Related papers  (2021-11-29T11:25:14Z)
• Consistent Depth of Moving Objects in Video [52.72092264848864]
  We present a method to estimate depth of a dynamic scene, containing arbitrary moving objects, from an ordinary video captured with a moving camera. We formulate this objective in a new test-time training framework where a depth-prediction CNN is trained in tandem with an auxiliary scene-flow prediction over the entire input video. We demonstrate accurate and temporally coherent results on a variety of challenging videos containing diverse moving objects (pets, people, cars) as well as camera motion.
  arXiv  Detail & Related papers  (2021-08-02T20:53:18Z)
• Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras [3.485767750936058]
  Multidimensional Vector is proposed to include the utilizable information generated in different dimensions and stages. The experiments of real fisheye images demonstrate that our solution achieves state-of-the-art accuracy while being real-time in practice.
  arXiv  Detail & Related papers  (2021-07-19T13:24:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.