PRISE: Demystifying Deep Lucas-Kanade with Strongly Star-Convex
Constraints for Multimodel Image Alignment
- URL: http://arxiv.org/abs/2303.11526v1
- Date: Tue, 21 Mar 2023 01:19:35 GMT
- Title: PRISE: Demystifying Deep Lucas-Kanade with Strongly Star-Convex
Constraints for Multimodel Image Alignment
- Authors: Yiqing Zhang, Xinming Huang, Ziming Zhang
- Abstract summary: The Lucas-Kanade (LK) method is a classic iterative homography estimation algorithm for image, but often suffers from poor local optimality especially when image pairs have distortions.
We present a novel Deep Star-Convexified Lucas-Kanade (RISE) method for image optimization.
- Score: 18.30521162275051
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Lucas-Kanade (LK) method is a classic iterative homography estimation
algorithm for image alignment, but often suffers from poor local optimality
especially when image pairs have large distortions. To address this challenge,
in this paper we propose a novel Deep Star-Convexified Lucas-Kanade (PRISE)
method for multimodel image alignment by introducing strongly star-convex
constraints into the optimization problem. Our basic idea is to enforce the
neural network to approximately learn a star-convex loss landscape around the
ground truth give any data to facilitate the convergence of the LK method to
the ground truth through the high dimensional space defined by the network.
This leads to a minimax learning problem, with contrastive (hinge) losses due
to the definition of strong star-convexity that are appended to the original
loss for training. We also provide an efficient sampling based algorithm to
leverage the training cost, as well as some analysis on the quality of the
solutions from PRISE. We further evaluate our approach on benchmark datasets
such as MSCOCO, GoogleEarth, and GoogleMap, and demonstrate state-of-the-art
results, especially for small pixel errors. Code can be downloaded from
https://github.com/Zhang-VISLab.
Related papers
- GLACE: Global Local Accelerated Coordinate Encoding [66.87005863868181]
Scene coordinate regression methods are effective in small-scale scenes but face significant challenges in large-scale scenes.
We propose GLACE, which integrates pre-trained global and local encodings and enables SCR to scale to large scenes with only a single small-sized network.
Our method achieves state-of-the-art results on large-scale scenes with a low-map-size model.
arXiv Detail & Related papers (2024-06-06T17:59:50Z) - Learning from small data sets: Patch-based regularizers in inverse
problems for image reconstruction [1.1650821883155187]
Recent advances in machine learning require a huge amount of data and computer capacity to train the networks.
Our paper addresses the issue of learning from small data sets by taking patches of very few images into account.
We show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods.
arXiv Detail & Related papers (2023-12-27T15:30:05Z) - Depth Monocular Estimation with Attention-based Encoder-Decoder Network
from Single Image [7.753378095194288]
Vision-based approaches have recently received much attention and can overcome these drawbacks.
In this work, we explore an extreme scenario in vision-based settings: estimate a depth map from one monocular image severely plagued by grid artifacts and blurry edges.
Our novel approach can find the focus of current image with minimal overhead and avoid losses of depth features.
arXiv Detail & Related papers (2022-10-24T23:01:25Z) - SDWNet: A Straight Dilated Network with Wavelet Transformation for Image
Deblurring [23.86692375792203]
Image deblurring is a computer vision problem that aims to recover a sharp image from a blurred image.
Our model uses dilated convolution to enable the obtainment of the large receptive field with high spatial resolution.
We propose a novel module using the wavelet transform, which effectively helps the network to recover clear high-frequency texture details.
arXiv Detail & Related papers (2021-10-12T07:58:10Z) - InfinityGAN: Towards Infinite-Resolution Image Synthesis [92.40782797030977]
We present InfinityGAN, a method to generate arbitrary-resolution images.
We show how it trains and infers patch-by-patch seamlessly with low computational resources.
arXiv Detail & Related papers (2021-04-08T17:59:30Z) - SDAN: Squared Deformable Alignment Network for Learning Misaligned
Optical Zoom [5.202871995038932]
Deep Neural Network (DNN) based super-resolution algorithms have greatly improved the quality of the generated images.
These algorithms often yield significant artifacts when dealing with real-world super-resolution problems.
We introduce a Squared Deformable Alignment Network (SDAN) to address this issue.
arXiv Detail & Related papers (2021-04-02T01:58:00Z) - Image Restoration by Deep Projected GSURE [115.57142046076164]
Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution.
We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
arXiv Detail & Related papers (2021-02-04T08:52:46Z) - LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization [54.77498358487812]
LM-Reloc is a novel approach for visual relocalization based on direct image alignment.
We propose a loss formulation inspired by the classical Levenberg-Marquardt algorithm to train LM-Net.
arXiv Detail & Related papers (2020-10-13T12:15:20Z) - A deep primal-dual proximal network for image restoration [8.797434238081372]
We design a deep network, named DeepPDNet, built from primal-dual iterations associated with the minimization of a standard penalized likelihood with an analysis prior.
Two different learning strategies: "Full learning" and "Partial learning" are proposed, the first one is the most efficient numerically.
Extensive results show that the proposed DeepPDNet demonstrates excellent performance on the MNIST and the more complex BSD68, BSD100, and SET14 datasets for image restoration and single image super-resolution task.
arXiv Detail & Related papers (2020-07-02T08:29:52Z) - The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors.
We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS.
To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
arXiv Detail & Related papers (2020-05-16T08:17:44Z) - PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of
Generative Models [77.32079593577821]
PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature.
Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
arXiv Detail & Related papers (2020-03-08T16:44:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.