DUT: Learning Video Stabilization by Simply Watching Unstable Videos

• URL: http://arxiv.org/abs/2011.14574v2
• Date: Tue, 1 Dec 2020 02:40:19 GMT
• Title: DUT: Learning Video Stabilization by Simply Watching Unstable Videos
• Authors: Yufei Xu, Jing Zhang, Stephen J. Maybank, Dacheng Tao
• Abstract summary: We propose a Deep Unsupervised Trajectory-based stabilization framework (DUT) DUT makes the first attempt to stabilize unstable videos by explicitly estimating and smoothing trajectories in an unsupervised deep learning manner. Experiment results on public benchmarks show that DUT outperforms representative state-of-the-art methods both qualitatively and quantitatively.
• Score: 86.88635774560017
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a Deep Unsupervised Trajectory-based stabilization framework (DUT) in this paper. Traditional stabilizers focus on trajectory-based smoothing, which is controllable but fragile in occluded and textureless cases regarding the usage of hand-crafted features. On the other hand, previous deep video stabilizers directly generate stable videos in a supervised manner without explicit trajectory estimation, which is robust but less controllable and the appropriate paired data are hard to obtain. To construct a controllable and robust stabilizer, DUT makes the first attempt to stabilize unstable videos by explicitly estimating and smoothing trajectories in an unsupervised deep learning manner, which is composed of a DNN-based keypoint detector and motion estimator to generate grid-based trajectories, and a DNN-based trajectory smoother to stabilize videos. We exploit both the nature of continuity in motion and the consistency of keypoints and grid vertices before and after stabilization for unsupervised training. Experiment results on public benchmarks show that DUT outperforms representative state-of-the-art methods both qualitatively and quantitatively.

Related papers

• Towards Stable 3D Object Detection [64.49059005467817]
  Stability Index (SI) is a new metric that can comprehensively evaluate the stability of 3D detectors in terms of confidence, box localization, extent, and heading. To help models improve their stability, we introduce a general and effective training strategy, called Prediction Consistency Learning (PCL) PCL essentially encourages the prediction consistency of the same objects under different timestamps and augmentations, leading to enhanced detection stability.
  arXiv  Detail & Related papers  (2024-07-05T07:17:58Z)
• Learning to Boost the Performance of Stable Nonlinear Systems [0.0]
  We tackle the performance-boosting problem with closed-loop stability guarantees. Our methods enable learning over arbitrarily deep neural network classes of performance-boosting controllers for stable nonlinear systems.
  arXiv  Detail & Related papers  (2024-05-01T21:11:29Z)
• GPU-accelerated SIFT-aided source identification of stabilized videos [63.084540168532065]
  We exploit the parallelization capabilities of Graphics Processing Units (GPUs) in the framework of stabilised frames inversion. We propose to exploit SIFT features. to estimate the camera momentum and %to identify less stabilized temporal segments. Experiments confirm the effectiveness of the proposed approach in reducing the required computational time and improving the source identification accuracy.
  arXiv  Detail & Related papers  (2022-07-29T07:01:31Z)
• Robust Stability of Neural-Network Controlled Nonlinear Systems with Parametric Variability [2.0199917525888895]
  We develop a theory for stability and stabilizability of a class of neural-network controlled nonlinear systems. For computing such a robust stabilizing NN controller, a stability guaranteed training (SGT) is also proposed.
  arXiv  Detail & Related papers  (2021-09-13T05:09:30Z)
• Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization [82.56853587380168]
  Warping-based video stabilizers smooth camera trajectory by constraining each pixel's displacement and warp frames from unstable ones. OVS can be integrated into existing warping-based stabilizers as a plug-and-play module to significantly improve the cropping ratio of the stabilized results.
  arXiv  Detail & Related papers  (2021-08-20T08:07:47Z)
• Neural Re-rendering for Full-frame Video Stabilization [144.9918806873405]
  We present an algorithm for full-frame video stabilization by first estimating dense warp fields. Full-frame stabilized frames can then be synthesized by fusing warped contents from neighboring frames.
  arXiv  Detail & Related papers  (2021-02-11T18:59:45Z)
• Deep Motion Blind Video Stabilization [4.544151613454639]
  This work aims to declutter this over-complicated formulation of video stabilization with the help of a novel dataset. We successfully learn motion blind full-frame video stabilization through employing strictly conventional generative techniques. Our method achieves $sim3times$ speed-up over the currently available fastest video stabilization methods.
  arXiv  Detail & Related papers  (2020-11-19T07:26:06Z)
• Diagnosing and Preventing Instabilities in Recurrent Video Processing [23.39527368516591]
  We show that video stability models tend to fail catastrophically at inference time on long visualizations. We introduce a diagnostic tool which produces adversarial input sequences optimized to trigger instabilities. We then introduce Stable Rank Normalization of the Layers (SRNL), a new algorithm that enforces these constraints.
  arXiv  Detail & Related papers  (2020-10-10T21:39:28Z)
• Robust Model-Free Learning and Control without Prior Knowledge [1.14219428942199]
  We present a model-free control algorithm that robustly learn and stabilize an unknown discrete-time linear system. The controller does not require any prior knowledge of the system dynamics, disturbances, or noise. We will conclude with simulation results that show that despite the generality and simplicity, the controller demonstrates good closed-loop performance.
  arXiv  Detail & Related papers  (2020-10-01T05:43:33Z)
• Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory [85.29718245299341]
  We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR) We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We propose an efficient data dependent algorithm -- textsceXploration -- that with high probability quickly identifies a stabilizing controller.
  arXiv  Detail & Related papers  (2020-06-19T08:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.