Region-wise matching for image inpainting based on adaptive weighted low-rank decomposition

• URL: http://arxiv.org/abs/2303.12421v1
• Date: Wed, 22 Mar 2023 09:38:34 GMT
• Title: Region-wise matching for image inpainting based on adaptive weighted low-rank decomposition
• Authors: Shenghai Liao, Xuya Liu, Ruyi Han, Shujun Fu, Yuanfeng Zhou and Yuliang Li
• Abstract summary: Digital image inpainting is a problem, inferring the content in the missing (unknown) region to agree with the known region data. Low-rank and nonlocal self-similarity are two important priors for image inpainting. We propose a region-wise (RwM) algorithm by dividing the neighborhood of a target patch into multiple subregions and then search the most similar one within each subregion.
• Score: 4.323678254948305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Digital image inpainting is an interpolation problem, inferring the content in the missing (unknown) region to agree with the known region data such that the interpolated result fulfills some prior knowledge. Low-rank and nonlocal self-similarity are two important priors for image inpainting. Based on the nonlocal self-similarity assumption, an image is divided into overlapped square target patches (submatrices) and the similar patches of any target patch are reshaped as vectors and stacked into a patch matrix. Such a patch matrix usually enjoys a property of low rank or approximately low rank, and its missing entries are recoveried by low-rank matrix approximation (LRMA) algorithms. Traditionally, $n$ nearest neighbor similar patches are searched within a local window centered at a target patch. However, for an image with missing lines, the generated patch matrix is prone to having entirely-missing rows such that the downstream low-rank model fails to reconstruct it well. To address this problem, we propose a region-wise matching (RwM) algorithm by dividing the neighborhood of a target patch into multiple subregions and then search the most similar one within each subregion. A non-convex weighted low-rank decomposition (NC-WLRD) model for LRMA is also proposed to reconstruct all degraded patch matrices grouped by the proposed RwM algorithm. We solve the proposed NC-WLRD model by the alternating direction method of multipliers (ADMM) and analyze the convergence in detail. Numerous experiments on line inpainting (entire-row/column missing) demonstrate the superiority of our method over other competitive inpainting algorithms. Unlike other low-rank-based matrix completion methods and inpainting algorithms, the proposed model NC-WLRD is also effective for removing random-valued impulse noise and structural noise (stripes).

Related papers

• Learning to Rank Patches for Unbiased Image Redundancy Reduction [80.93989115541966]
  Images suffer from heavy spatial redundancy because pixels in neighboring regions are spatially correlated. Existing approaches strive to overcome this limitation by reducing less meaningful image regions. We propose a self-supervised framework for image redundancy reduction called Learning to Rank Patches.
  arXiv  Detail & Related papers  (2024-03-31T13:12:41Z)
• RecDiffusion: Rectangling for Image Stitching with Diffusion Models [53.824503710254206]
  We introduce a novel diffusion-based learning framework, textbfRecDiffusion, for image stitching rectangling. This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary.
  arXiv  Detail & Related papers  (2024-03-28T06:22:45Z)
• Spectral Entry-wise Matrix Estimation for Low-Rank Reinforcement Learning [53.445068584013896]
  We study matrix estimation problems arising in reinforcement learning (RL) with low-rank structure. In low-rank bandits, the matrix to be recovered specifies the expected arm rewards, and for low-rank Markov Decision Processes (MDPs), it may for example characterize the transition kernel of the MDP. We show that simple spectral-based matrix estimation approaches efficiently recover the singular subspaces of the matrix and exhibit nearly-minimal entry-wise error.
  arXiv  Detail & Related papers  (2023-10-10T17:06:41Z)
• Learning the Positions in CountSketch [49.57951567374372]
  We consider sketching algorithms which first compress data by multiplication with a random sketch matrix, and then apply the sketch to quickly solve an optimization problem. In this work, we propose the first learning-based algorithms that also optimize the locations of the non-zero entries.
  arXiv  Detail & Related papers  (2023-06-11T07:28:35Z)
• Multi-channel Nuclear Norm Minus Frobenius Norm Minimization for Color Image Denoising [9.20787253404652]
  One traditional strategy is to convert the RGB image to a less correlated color space and denoise each channel of the new space separately. This paper proposes a new multi-channel optimization model for color image denoising under the nuclear norm minus Frobenius norm minimization framework. Experimental results on both synthetic and real noise datasets demonstrate the proposed model outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2022-09-16T04:10:29Z)
• Adaptive Non-linear Filtering Technique for Image Restoration [0.0]
  Decisionbased nonlinear algorithm for elimination of band lines, drop lines, mark, band lost and impulses in images is presented. Algorithm performs two simultaneous operations, namely, detection of corrupted pixels and evaluation of new pixels for replacing the corrupted pixels.
  arXiv  Detail & Related papers  (2022-04-20T08:36:59Z)
• Manifold-Inspired Single Image Interpolation [17.304301226838614]
  Many approaches to single image use manifold models to exploit semi-local similarity. aliasing in the input image makes it challenging for both parts. We propose a carefully-designed adaptive technique to remove aliasing in severely aliased regions. This technique enables reliable identification of similar patches even in the presence of strong aliasing.
  arXiv  Detail & Related papers  (2021-07-31T04:29:05Z)
• Inpainting Transformer for Anomaly Detection [0.0]
  Inpainting Transformer (InTra) is trained to inpaint covered patches in a large sequence of image patches. InTra achieves better than state-of-the-art results on the MVTec AD dataset for detection and localization.
  arXiv  Detail & Related papers  (2021-04-28T17:27:44Z)
• Image Denoising by Gaussian Patch Mixture Model and Low Rank Patches [0.0]
  A new method is proposed by solving two issues: how to improve similar patches matching accuracy and build an appropriate low rank matrix approximation model for Gaussian noise. By solving the two issues, experimental results are reported to show that the proposed approach outperforms the state-of-the-art denoising methods.
  arXiv  Detail & Related papers  (2020-11-20T09:28:04Z)
• LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization [54.77498358487812]
  LM-Reloc is a novel approach for visual relocalization based on direct image alignment. We propose a loss formulation inspired by the classical Levenberg-Marquardt algorithm to train LM-Net.
  arXiv  Detail & Related papers  (2020-10-13T12:15:20Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.