StabStitch++: Unsupervised Online Video Stitching with Spatiotemporal Bidirectional Warps

• URL: http://arxiv.org/abs/2505.05001v1
• Date: Thu, 08 May 2025 07:12:23 GMT
• Title: StabStitch++: Unsupervised Online Video Stitching with Spatiotemporal Bidirectional Warps
• Authors: Lang Nie, Chunyu Lin, Kang Liao, Yun Zhang, Shuaicheng Liu, Yao Zhao,
• Abstract summary: We retarget video stitching to an emerging issue, named warping shake, which unveils the temporal content shakes induced by sequentially unsmooth warps when extending image stitching to video stitching.<n>To address this issue, we propose StabStitch++, a novel video stitching framework to realize spatial stitching and temporal stabilization with unsupervised learning simultaneously.
• Score: 81.8786100662034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We retarget video stitching to an emerging issue, named warping shake, which unveils the temporal content shakes induced by sequentially unsmooth warps when extending image stitching to video stitching. Even if the input videos are stable, the stitched video can inevitably cause undesired warping shakes and affect the visual experience. To address this issue, we propose StabStitch++, a novel video stitching framework to realize spatial stitching and temporal stabilization with unsupervised learning simultaneously. First, different from existing learning-based image stitching solutions that typically warp one image to align with another, we suppose a virtual midplane between original image planes and project them onto it. Concretely, we design a differentiable bidirectional decomposition module to disentangle the homography transformation and incorporate it into our spatial warp, evenly spreading alignment burdens and projective distortions across two views. Then, inspired by camera paths in video stabilization, we derive the mathematical expression of stitching trajectories in video stitching by elaborately integrating spatial and temporal warps. Finally, a warp smoothing model is presented to produce stable stitched videos with a hybrid loss to simultaneously encourage content alignment, trajectory smoothness, and online collaboration. Compared with StabStitch that sacrifices alignment for stabilization, StabStitch++ makes no compromise and optimizes both of them simultaneously, especially in the online mode. To establish an evaluation benchmark and train the learning framework, we build a video stitching dataset with a rich diversity in camera motions and scenes. Experiments exhibit that StabStitch++ surpasses current solutions in stitching performance, robustness, and efficiency, offering compelling advancements in this field by building a real-time online video stitching system.

Related papers

• Video Demoireing using Focused-Defocused Dual-Camera System [21.59133575445115]
  Existing demoireing methods rely on single-camera image/video processing.<n>We propose a dual-camera framework that captures synchronized videos of the same scene.<n>We use the defocused video to help distinguish moire patterns from real texture, so as to guide the demoireing of the focused video.
  arXiv  Detail & Related papers  (2025-08-05T13:49:49Z)
• GaVS: 3D-Grounded Video Stabilization via Temporally-Consistent Local Reconstruction and Rendering [54.489285024494855]
  Video stabilization is pivotal for video processing, as it removes unwanted shakiness while preserving the original user motion intent.<n>Existing approaches, depending on the domain they operate, suffer from several issues that degrade the user experience.<n>We introduce textbfGaVS, a novel 3D-grounded approach that reformulates video stabilization as a temporally-consistent local reconstruction and rendering' paradigm.
  arXiv  Detail & Related papers  (2025-06-30T15:24:27Z)
• Beyond Wide-Angle Images: Structure-to-Detail Video Portrait Correction via Unsupervised Spatiotemporal Adaptation [30.50527916658749]
  We propose a structure-to-detail portrait correction model named ImagePC.<n>It integrates the long-range awareness of the transformer and multi-step denoising of diffusion models into a unified framework.<n>Considering the high cost of obtaining video labels, we then repurpose ImagePC for unlabeled wide-angle videos (termed VideoPC)
  arXiv  Detail & Related papers  (2025-04-01T03:49:59Z)
• WildVidFit: Video Virtual Try-On in the Wild via Image-Based Controlled Diffusion Models [132.77237314239025]
  Video virtual try-on aims to generate realistic sequences that maintain garment identity and adapt to a person's pose and body shape in source videos. Traditional image-based methods, relying on warping and blending, struggle with complex human movements and occlusions. We reconceptualize video try-on as a process of generating videos conditioned on garment descriptions and human motion. Our solution, WildVidFit, employs image-based controlled diffusion models for a streamlined, one-stage approach.
  arXiv  Detail & Related papers  (2024-07-15T11:21:03Z)
• Eliminating Warping Shakes for Unsupervised Online Video Stitching [77.96880800111498]
  We retarget video stitching to an emerging issue, named warping shake, when extending image stitching to video stitching. We propose StabStitch to simultaneously realize video stitching and video stabilization in a unified unsupervised learning framework. Compared with existing stitching solutions, StabStitch exhibits significant superiority in scene robustness and inference speed in addition to stitching and stabilization performance.
  arXiv  Detail & Related papers  (2024-03-11T02:05:31Z)
• Fast Full-frame Video Stabilization with Iterative Optimization [21.962533235492625]
  We propose an iterative optimization-based learning approach using synthetic datasets for video stabilization. We develop a two-level (coarse-to-fine) stabilizing algorithm based on the probabilistic flow field. We take a divide-and-conquer approach and propose a novel multiframe fusion strategy to render full-frame stabilized views.
  arXiv  Detail & Related papers  (2023-07-24T13:24:19Z)
• Video Demoireing with Relation-Based Temporal Consistency [68.20281109859998]
  Moire patterns, appearing as color distortions, severely degrade image and video qualities when filming a screen with digital cameras. We study how to remove such undesirable moire patterns in videos, namely video demoireing.
  arXiv  Detail & Related papers  (2022-04-06T17:45:38Z)
• Restoration of Video Frames from a Single Blurred Image with Motion Understanding [69.90724075337194]
  We propose a novel framework to generate clean video frames from a single motion-red image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors. Our framework is based on anblur-decoder structure with spatial transformer network modules.
  arXiv  Detail & Related papers  (2021-04-19T08:32:57Z)
• Aligning Videos in Space and Time [36.77248894563779]
  We propose a novel alignment procedure that learns such correspondence in space and time via cross video cycle-consistency. Our experiments on the Penn Action and Pouring datasets demonstrate that the proposed method can successfully learn to correspond semantically similar patches across videos.
  arXiv  Detail & Related papers  (2020-07-09T02:30:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.