RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

• URL: http://arxiv.org/abs/2301.01661v2
• Date: Tue, 5 Sep 2023 04:20:07 GMT
• Title: RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning
• Authors: Kang Liao, Lang Nie, Chunyu Lin, Zishuo Zheng, Yao Zhao
• Abstract summary: We propose a new learning model, i.e., Rectangling Rectification Network (RecRecNet) Our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations.
• Score: 62.86400614141706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The wide-angle lens shows appealing applications in VR technologies, but it introduces severe radial distortion into its captured image. To recover the realistic scene, previous works devote to rectifying the content of the wide-angle image. However, such a rectification solution inevitably distorts the image boundary, which changes related geometric distributions and misleads the current vision perception models. In this work, we explore constructing a win-win representation on both content and boundary by contributing a new learning model, i.e., Rectangling Rectification Network (RecRecNet). In particular, we propose a thin-plate spline (TPS) module to formulate the non-linear and non-rigid transformation for rectangling images. By learning the control points on the rectified image, our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. To relieve the complexity of structure approximation, we then inspire our RecRecNet to learn the gradual deformation rules with a DoF (Degree of Freedom)-based curriculum learning. By increasing the DoF in each curriculum stage, namely, from similarity transformation (4-DoF) to homography transformation (8-DoF), the network is capable of investigating more detailed deformations, offering fast convergence on the final rectangling task. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations. The code and dataset are available at https://github.com/KangLiao929/RecRecNet.

Related papers

• RecDiffusion: Rectangling for Image Stitching with Diffusion Models [53.824503710254206]
  We introduce a novel diffusion-based learning framework, textbfRecDiffusion, for image stitching rectangling. This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary.
  arXiv  Detail & Related papers  (2024-03-28T06:22:45Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
  We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique. First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion. To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
  arXiv  Detail & Related papers  (2023-02-16T10:40:55Z)
• Geo-SIC: Learning Deformable Geometric Shapes in Deep Image Classifiers [8.781861951759948]
  This paper presents Geo-SIC, the first deep learning model to learn deformable shapes in a deformation space for an improved performance of image classification. We introduce a newly designed framework that (i) simultaneously derives features from both image and latent shape spaces with large intra-class variations. We develop a boosted classification network, equipped with an unsupervised learning of geometric shape representations.
  arXiv  Detail & Related papers  (2022-10-25T01:55:17Z)
• Deep Rectangling for Image Stitching: A Learning Baseline [57.76737888499145]
  We build the first image stitching rectangling dataset with a large diversity in irregular boundaries and scenes. Experiments demonstrate our superiority over traditional methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-03-08T03:34:10Z)
• A training-free recursive multiresolution framework for diffeomorphic deformable image registration [6.929709872589039]
  We propose a novel diffeomorphic training-free approach for deformable image registration. The proposed architecture is simple in design. The moving image is warped successively at each resolution and finally aligned to the fixed image. The entire system is end-to-end and optimized for each pair of images from scratch.
  arXiv  Detail & Related papers  (2022-02-01T15:17:17Z)
• Stochastic Planner-Actor-Critic for Unsupervised Deformable Image Registration [33.72954116727303]
  We present a novel reinforcement learning-based framework that performs step-wise registration of medical images with large deformations. We evaluate our method on several 2D and 3D medical image datasets, some of which contain large deformations.
  arXiv  Detail & Related papers  (2021-12-14T14:08:56Z)
• Image Deformation Estimation via Multi-Objective Optimization [13.159751065619544]
  Free-form deformation model can represent a wide range of non-rigid deformations by manipulating a control point lattice over the image. It is challenging to fit the model directly to the deformed image for deformation estimation because of the complexity of the fitness landscape.
  arXiv  Detail & Related papers  (2021-06-08T06:52:12Z)
• Wide-angle Image Rectification: A Survey [86.36118799330802]
  wide-angle images contain distortions that violate the assumptions underlying pinhole camera models. Image rectification, which aims to correct these distortions, can solve these problems. We present a detailed description and discussion of the camera models used in different approaches. Next, we review both traditional geometry-based image rectification methods and deep learning-based methods.
  arXiv  Detail & Related papers  (2020-10-30T17:28:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.