Image Stitching and Rectification for Hand-Held Cameras

• URL: http://arxiv.org/abs/2008.09229v1
• Date: Thu, 20 Aug 2020 23:31:23 GMT
• Title: Image Stitching and Rectification for Hand-Held Cameras
• Authors: Bingbing Zhuang and Quoc-Huy Tran
• Abstract summary: We derive a new differential homography that can account for the scanline-varying camera poses in Rolling Shutter (RS) cameras. We show superior performance over state-of-the-art methods both in RS image stitching and rectification, especially for images captured by hand-held shaking cameras.
• Score: 17.694946451997815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we derive a new differential homography that can account for the scanline-varying camera poses in Rolling Shutter (RS) cameras, and demonstrate its application to carry out RS-aware image stitching and rectification at one stroke. Despite the high complexity of RS geometry, we focus in this paper on a special yet common input -- two consecutive frames from a video stream, wherein the inter-frame motion is restricted from being arbitrarily large. This allows us to adopt simpler differential motion model, leading to a straightforward and practical minimal solver. To deal with non-planar scene and camera parallax in stitching, we further propose an RS-aware spatially-varying homography field in the principle of As-Projective-As-Possible (APAP). We show superior performance over state-of-the-art methods both in RS image stitching and rectification, especially for images captured by hand-held shaking cameras.

Related papers

• Single Image Rolling Shutter Removal with Diffusion Models [46.57721145372241]
  We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction. In this work, we present an image-to-motion'' framework via diffusion techniques, with a designed patch-attention module. In addition, we present the RS-Real dataset, comprised of captured RS frames alongside their corresponding Global Shutter (GS) ground-truth pairs.
  arXiv  Detail & Related papers  (2024-07-03T08:25:02Z)
• Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction [54.00007868515432]
  Existing methods face challenges in estimating the accurate correction field due to the uniform velocity assumption. We propose a geometry-based Quadratic Rolling Shutter (QRS) motion solver, which precisely estimates the high-order correction field of individual pixels. Our method surpasses the state-of-the-art by +4.98, +0.77, and +4.33 of PSNR on Carla-RS, Fastec-RS, and BS-RSC datasets, respectively.
  arXiv  Detail & Related papers  (2023-03-31T15:09:18Z)
• Rolling Shutter Inversion: Bring Rolling Shutter Images to High Framerate Global Shutter Video [111.08121952640766]
  This paper presents a novel deep-learning based solution to the RS temporal super-resolution problem. By leveraging the multi-view geometry relationship of the RS imaging process, our framework successfully achieves high framerate GS generation. Our method can produce high-quality GS image sequences with rich details, outperforming the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-10-06T16:47:12Z)
• Video frame interpolation for high dynamic range sequences captured with dual-exposure sensors [24.086089662881044]
  Video frame (VFI) enables many important applications that might involve the temporal domain. One of the key challenges is handling high dynamic range scenes in the presence of complex motion.
  arXiv  Detail & Related papers  (2022-06-19T20:29:34Z)
• Context-Aware Video Reconstruction for Rolling Shutter Cameras [52.28710992548282]
  In this paper, we propose a context-aware GS video reconstruction architecture. We first estimate the bilateral motion field so that the pixels of the two RS frames are warped to a common GS frame. Then, a refinement scheme is proposed to guide the GS frame synthesis along with bilateral occlusion masks to produce high-fidelity GS video frames.
  arXiv  Detail & Related papers  (2022-05-25T17:05:47Z)
• Learning Adaptive Warping for Real-World Rolling Shutter Correction [52.45689075940234]
  This paper proposes the first real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video. Mobile devices in the consumer market with CMOS-based sensors for video capture often result in rolling shutter effects when relative movements occur during the video acquisition process.
  arXiv  Detail & Related papers  (2022-04-29T05:13:50Z)
• Bringing Rolling Shutter Images Alive with Dual Reversed Distortion [75.78003680510193]
  Rolling shutter (RS) distortion can be interpreted as the result of picking a row of pixels from instant global shutter (GS) frames over time. We develop a novel end-to-end model, IFED, to generate dual optical flow sequence through iterative learning of the velocity field during the RS time.
  arXiv  Detail & Related papers  (2022-03-12T14:57:49Z)
• Deep network for rolling shutter rectification [25.170821013431958]
  We propose an end-to-end deep neural network for the challenging task of single image RS rectification. Our network consists of a motion block, a trajectory module, a row block, an RS rectification module and an RS regeneration module. Experiments on synthetic and real datasets reveal that our network outperforms prior art both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2021-12-12T06:40:34Z)
• From two rolling shutters to one global shutter [57.431998188805665]
  We explore a surprisingly simple camera configuration that makes it possible to undo the rolling shutter distortion. Such a setup is easy and cheap to build and it possesses the geometric constraints needed to correct rolling shutter distortion. We derive equations that describe the underlying geometry for general and special motions and present an efficient method for finding their solutions.
  arXiv  Detail & Related papers  (2020-06-02T22:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.