Semi-Supervised Coupled Thin-Plate Spline Model for Rotation Correction and Beyond

• URL: http://arxiv.org/abs/2401.13432v2
• Date: Tue, 18 Jun 2024 10:29:39 GMT
• Title: Semi-Supervised Coupled Thin-Plate Spline Model for Rotation Correction and Beyond
• Authors: Lang Nie, Chunyu Lin, Kang Liao, Shuaicheng Liu, Yao Zhao,
• Abstract summary: We propose CoupledTPS, which iteratively couples multiple TPS with limited control points into a more flexible and powerful transformation. In light of the laborious annotation cost, we develop a semi-supervised learning scheme to improve warping quality by exploiting unlabeled data. Experiments demonstrate the superiority and universality of CoupledTPS over the existing state-of-the-art solutions for rotation correction.
• Score: 84.56978780892783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Thin-plate spline (TPS) is a principal warp that allows for representing elastic, nonlinear transformation with control point motions. With the increase of control points, the warp becomes increasingly flexible but usually encounters a bottleneck caused by undesired issues, e.g., content distortion. In this paper, we explore generic applications of TPS in single-image-based warping tasks, such as rotation correction, rectangling, and portrait correction. To break this bottleneck, we propose the coupled thin-plate spline model (CoupledTPS), which iteratively couples multiple TPS with limited control points into a more flexible and powerful transformation. Concretely, we first design an iterative search to predict new control points according to the current latent condition. Then, we present the warping flow as a bridge for the coupling of different TPS transformations, effectively eliminating interpolation errors caused by multiple warps. Besides, in light of the laborious annotation cost, we develop a semi-supervised learning scheme to improve warping quality by exploiting unlabeled data. It is formulated through dual transformation between the searched control points of unlabeled data and its graphic augmentation, yielding an implicit correction consistency constraint. Finally, we collect massive unlabeled data to exhibit the benefit of our semi-supervised scheme in rotation correction. Extensive experiments demonstrate the superiority and universality of CoupledTPS over the existing state-of-the-art (SoTA) solutions for rotation correction and beyond. The code and data are available at https://github.com/nie-lang/CoupledTPS.

Related papers

• Projected Entangled Pair States with flexible geometry [0.0]
  Projected Entangled Pair States (PEPS) are a class of quantum many-body states that generalize Matrix Product States for one-dimensional systems to higher dimensions. PEPS have advanced understanding of strongly correlated systems, especially in two dimensions, e.g., quantum spin liquids. We present a PEPS algorithm to simulate low-energy states and dynamics defined on arbitrary, fluctuating, and densely connected graphs.
  arXiv  Detail & Related papers  (2024-07-30T19:03:52Z)
• Entropy Transformer Networks: A Learning Approach via Tangent Bundle Data Manifold [8.893886200299228]
  This paper focuses on an accurate and fast approach for image transformation employed in the design of CNN architectures. A novel Entropy STN (ESTN) is proposed that interpolates on the data manifold distributions. Experiments on challenging benchmarks show that the proposed ESTN can improve predictive accuracy over a range of computer vision tasks.
  arXiv  Detail & Related papers  (2023-07-24T04:21:51Z)
• TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition [78.67283660198403]
  Text irregularities pose significant challenges to scene text recognizers. TPS++ is an attention-enhanced TPS transformation that incorporates the attention mechanism to text rectification. It consistently improves the recognition and achieves state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2023-05-09T10:16:43Z)
• M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion [19.862336286338564]
  In federated learning, model updates must be compressed so as to minimize the loss in accuracy resulting from a communication constraint. This paper proposes emph$bf M$-magnitude weighted $L_bf 2$ distortion + $bf 2$ degrees of freedom'' (M22) algorithm, a rate-distortion inspired approach to gradient compression.
  arXiv  Detail & Related papers  (2023-01-23T04:40:01Z)
• RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning [62.86400614141706]
  We propose a new learning model, i.e., Rectangling Rectification Network (RecRecNet) Our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2023-01-04T15:12:57Z)
• Graph Reasoning Transformer for Image Parsing [67.76633142645284]
  We propose a novel Graph Reasoning Transformer (GReaT) for image parsing to enable image patches to interact following a relation reasoning pattern. Compared to the conventional transformer, GReaT has higher interaction efficiency and a more purposeful interaction pattern. Results show that GReaT achieves consistent performance gains with slight computational overheads on the state-of-the-art transformer baselines.
  arXiv  Detail & Related papers  (2022-09-20T08:21:37Z)
• Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation [50.62685357414904]
  Video Panoptic coefficient (VPS) aims to generate coherent panoptic segmentation and track the identities of all pixels across video frames. We present HybridTracker, a lightweight and joint tracking model attempting to eliminate the limitations of the single tracker. Comprehensive experiments show that HybridTracker achieves superior performance than state-of-the-art methods on Cityscapes-VPS and VIPER datasets.
  arXiv  Detail & Related papers  (2022-03-02T16:21:55Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• Semi-Supervised Wide-Angle Portraits Correction by Multi-Scale Transformer [17.455782652441187]
  We propose a semi-supervised network for wide-angle portraits correction. Our network, named as Multi-Scale Swin-Unet (MS-Unet), is built upon the multi-scale swin transformer block (MSTB)
  arXiv  Detail & Related papers  (2021-09-14T09:40:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.