Adaptively Meshed Video Stabilization

• URL: http://arxiv.org/abs/2006.07820v1
• Date: Sun, 14 Jun 2020 06:51:23 GMT
• Title: Adaptively Meshed Video Stabilization
• Authors: Minda Zhao, Qiang Ling
• Abstract summary: This paper proposes an adaptively meshed method to stabilize a shaky video based on all of its feature trajectories and an adaptive blocking strategy. We estimate the mesh-based transformations of each frame by solving a two-stage optimization problem.
• Score: 32.68960056325736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video stabilization is essential for improving visual quality of shaky videos. The current video stabilization methods usually take feature trajectories in the background to estimate one global transformation matrix or several transformation matrices based on a fixed mesh, and warp shaky frames into their stabilized views. However, these methods may not model the shaky camera motion well in complicated scenes, such as scenes containing large foreground objects or strong parallax, and may result in notable visual artifacts in the stabilized videos. To resolve the above issues, this paper proposes an adaptively meshed method to stabilize a shaky video based on all of its feature trajectories and an adaptive blocking strategy. More specifically, we first extract feature trajectories of the shaky video and then generate a triangle mesh according to the distribution of the feature trajectories in each frame. Then transformations between shaky frames and their stabilized views over all triangular grids of the mesh are calculated to stabilize the shaky video. Since more feature trajectories can usually be extracted from all regions, including both background and foreground regions, a finer mesh will be obtained and provided for camera motion estimation and frame warping. We estimate the mesh-based transformations of each frame by solving a two-stage optimization problem. Moreover, foreground and background feature trajectories are no longer distinguished and both contribute to the estimation of the camera motion in the proposed optimization problem, which yields better estimation performance than previous works, particularly in challenging videos with large foreground objects or strong parallax.

Related papers

• Fast Full-frame Video Stabilization with Iterative Optimization [21.962533235492625]
  We propose an iterative optimization-based learning approach using synthetic datasets for video stabilization. We develop a two-level (coarse-to-fine) stabilizing algorithm based on the probabilistic flow field. We take a divide-and-conquer approach and propose a novel multiframe fusion strategy to render full-frame stabilized views.
  arXiv  Detail & Related papers  (2023-07-24T13:24:19Z)
• Minimum Latency Deep Online Video Stabilization [77.68990069996939]
  We present a novel camera path optimization framework for the task of online video stabilization. In this work, we adopt recent off-the-shelf high-quality deep motion models for motion estimation to recover the camera trajectory. Our approach significantly outperforms state-of-the-art online methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-12-05T07:37:32Z)
• Pyramid Feature Alignment Network for Video Deblurring [63.26197177542422]
  Video deblurring is a challenging task due to various causes of blurring. Traditional methods have considered how to utilize neighboring frames by the single-scale alignment for restoration. We propose a Pyramid Feature Alignment Network (PFAN) for video deblurring.
  arXiv  Detail & Related papers  (2022-03-28T07:54:21Z)
• Self-Supervised Real-time Video Stabilization [100.00816752529045]
  We propose a novel method of real-time video stabilization. It transforms a shaky video to a stabilized video as if it were stabilized via gimbals in real-time.
  arXiv  Detail & Related papers  (2021-11-10T22:49:56Z)
• Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes [14.384467317051831]
  We propose two novel approaches to deblurring videos by effectively aggregating information from multiple video frames. First, we present blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames. Second, for motion compensation, instead of aligning frames by warping with estimated motions, we use a pixel volume that contains candidate sharp pixels to resolve motion estimation errors.
  arXiv  Detail & Related papers  (2021-08-23T07:36:49Z)
• Restoration of Video Frames from a Single Blurred Image with Motion Understanding [69.90724075337194]
  We propose a novel framework to generate clean video frames from a single motion-red image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors. Our framework is based on anblur-decoder structure with spatial transformer network modules.
  arXiv  Detail & Related papers  (2021-04-19T08:32:57Z)
• Motion-blurred Video Interpolation and Extrapolation [72.3254384191509]
  We present a novel framework for deblurring, interpolating and extrapolating sharp frames from a motion-blurred video in an end-to-end manner. To ensure temporal coherence across predicted frames and address potential temporal ambiguity, we propose a simple, yet effective flow-based rule.
  arXiv  Detail & Related papers  (2021-03-04T12:18:25Z)
• Neural Re-rendering for Full-frame Video Stabilization [144.9918806873405]
  We present an algorithm for full-frame video stabilization by first estimating dense warp fields. Full-frame stabilized frames can then be synthesized by fusing warped contents from neighboring frames.
  arXiv  Detail & Related papers  (2021-02-11T18:59:45Z)
• Deep Motion Blind Video Stabilization [4.544151613454639]
  This work aims to declutter this over-complicated formulation of video stabilization with the help of a novel dataset. We successfully learn motion blind full-frame video stabilization through employing strictly conventional generative techniques. Our method achieves $sim3times$ speed-up over the currently available fastest video stabilization methods.
  arXiv  Detail & Related papers  (2020-11-19T07:26:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.