Parallax-Tolerant Unsupervised Deep Image Stitching

• URL: http://arxiv.org/abs/2302.08207v2
• Date: Sat, 22 Jul 2023 03:47:27 GMT
• Title: Parallax-Tolerant Unsupervised Deep Image Stitching
• Authors: Lang Nie, Chunyu Lin, Kang Liao, Shuaicheng Liu, Yao Zhao
• Abstract summary: We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique. First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion. To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
• Score: 57.76737888499145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional image stitching approaches tend to leverage increasingly complex geometric features (point, line, edge, etc.) for better performance. However, these hand-crafted features are only suitable for specific natural scenes with adequate geometric structures. In contrast, deep stitching schemes overcome the adverse conditions by adaptively learning robust semantic features, but they cannot handle large-parallax cases due to homography-based registration. To solve these issues, we propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique. First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion. It provides accurate alignment for overlapping regions and shape preservation for non-overlapping regions by joint optimization concerning alignment and distortion. Subsequently, to improve the generalization capability, we design a simple but effective iterative strategy to enhance the warp adaption in cross-dataset and cross-resolution applications. Finally, to further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks. Compared with existing methods, our solution is parallax-tolerant and free from laborious designs of complicated geometric features for specific scenes. Extensive experiments show our superiority over the SoTA methods, both quantitatively and qualitatively. The code is available at https://github.com/nie-lang/UDIS2.

Related papers

• RecDiffusion: Rectangling for Image Stitching with Diffusion Models [53.824503710254206]
  We introduce a novel diffusion-based learning framework, textbfRecDiffusion, for image stitching rectangling. This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary.
  arXiv  Detail & Related papers  (2024-03-28T06:22:45Z)
• Learning Residual Elastic Warps for Image Stitching under Dirichlet Boundary Condition [28.627775495233692]
  We suggest Recurrent Elastic Warps (REwarp) that address the problem with Dirichlet boundary condition. REwarp predicts a homography and a Thin-plate Spline (TPS) under the boundary constraint for discontinuity and hole-free image stitching. Our experiments show the favorable aligns and the competitive computational costs of REwarp compared to the existing stitching methods.
  arXiv  Detail & Related papers  (2023-09-04T07:26:42Z)
• Collaborative Blind Image Deblurring [15.555393702795076]
  We show that when extracting patches of similar underlying blur is possible, jointly processing the stack of patches yields superior accuracy than handling them separately. We present three practical patch extraction strategies for image sharpening, camera shake removal and optical aberration correction, and validate the proposed approach on both synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2023-05-25T13:14:29Z)
• RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning [62.86400614141706]
  We propose a new learning model, i.e., Rectangling Rectification Network (RecRecNet) Our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2023-01-04T15:12:57Z)
• Pyramid Feature Alignment Network for Video Deblurring [63.26197177542422]
  Video deblurring is a challenging task due to various causes of blurring. Traditional methods have considered how to utilize neighboring frames by the single-scale alignment for restoration. We propose a Pyramid Feature Alignment Network (PFAN) for video deblurring.
  arXiv  Detail & Related papers  (2022-03-28T07:54:21Z)
• Deep Rectangling for Image Stitching: A Learning Baseline [57.76737888499145]
  We build the first image stitching rectangling dataset with a large diversity in irregular boundaries and scenes. Experiments demonstrate our superiority over traditional methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-03-08T03:34:10Z)
• Pixel-wise Deep Image Stitching [21.824319551526294]
  Image stitching aims at stitching the images taken from different viewpoints into an image with a wider field of view. Existing methods warp the target image to the reference image using the estimated warp function. We propose a novel deep image stitching framework exploiting the pixel-wise warp field to handle the large-parallax problem.
  arXiv  Detail & Related papers  (2021-12-12T07:28:48Z)
• Manifold-Inspired Single Image Interpolation [17.304301226838614]
  Many approaches to single image use manifold models to exploit semi-local similarity. aliasing in the input image makes it challenging for both parts. We propose a carefully-designed adaptive technique to remove aliasing in severely aliased regions. This technique enables reliable identification of similar patches even in the presence of strong aliasing.
  arXiv  Detail & Related papers  (2021-07-31T04:29:05Z)
• Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation [38.95610086309832]
  Homography estimation is an important task in computer vision, such as image stitching, video stabilization, and camera calibration. Traditional homography estimation methods depend on the quantity and distribution of feature points, leading to poor robustness in textureless scenes. We propose a contextual correlation layer, which can capture the long-range correlation on feature maps and flexibly be bridged in a learning framework. We equip our network with depth perception capability, by introducing a novel depth-aware shape-preserved loss.
  arXiv  Detail & Related papers  (2021-07-06T10:33:12Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.