Region Normalization for Image Inpainting
- URL: http://arxiv.org/abs/1911.10375v2
- Date: Tue, 14 Mar 2023 11:38:46 GMT
- Title: Region Normalization for Image Inpainting
- Authors: Tao Yu, Zongyu Guo, Xin Jin, Shilin Wu, Zhibo Chen, Weiping Li,
Zhizheng Zhang, Sen Liu
- Abstract summary: Feature Normalization (FN) is an important technique to help neural network training, which typically normalizes features across spatial dimensions.
In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training.
We propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation.
- Score: 52.17610250998762
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Feature Normalization (FN) is an important technique to help neural network
training, which typically normalizes features across spatial dimensions. Most
previous image inpainting methods apply FN in their networks without
considering the impact of the corrupted regions of the input image on
normalization, e.g. mean and variance shifts. In this work, we show that the
mean and variance shifts caused by full-spatial FN limit the image inpainting
network training and we propose a spatial region-wise normalization named
Region Normalization (RN) to overcome the limitation. RN divides spatial pixels
into different regions according to the input mask, and computes the mean and
variance in each region for normalization. We develop two kinds of RN for our
image inpainting network: (1) Basic RN (RN-B), which normalizes pixels from the
corrupted and uncorrupted regions separately based on the original inpainting
mask to solve the mean and variance shift problem; (2) Learnable RN (RN-L),
which automatically detects potentially corrupted and uncorrupted regions for
separate normalization, and performs global affine transformation to enhance
their fusion. We apply RN-B in the early layers and RN-L in the latter layers
of the network respectively. Experiments show that our method outperforms
current state-of-the-art methods quantitatively and qualitatively. We further
generalize RN to other inpainting networks and achieve consistent performance
improvements. Our code is available at https://github.com/geekyutao/RN.
Related papers
- MrRegNet: Multi-resolution Mask Guided Convolutional Neural Network for Medical Image Registration with Large Deformations [6.919880141683284]
MrRegNet is a mask-guided encoder-decoder DCNN-based image registration method.
Image alignment accuracies are significantly improved at local regions guided by segmentation masks.
arXiv Detail & Related papers (2024-05-16T12:57:03Z) - A Prototype-Based Neural Network for Image Anomaly Detection and Localization [10.830337829732915]
This paper proposes ProtoAD, a prototype-based neural network for image anomaly detection and localization.
First, the patch features of normal images are extracted by a deep network pre-trained on nature images.
ProtoAD achieves competitive performance compared to the state-of-the-art methods with a higher inference speed.
arXiv Detail & Related papers (2023-10-04T04:27:16Z) - RePaint: Inpainting using Denoising Diffusion Probabilistic Models [161.74792336127345]
Free-form inpainting is the task of adding new content to an image in the regions specified by an arbitrary binary mask.
We propose RePaint: A Denoising Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
We validate our method for both faces and general-purpose image inpainting using standard and extreme masks.
arXiv Detail & Related papers (2022-01-24T18:40:15Z) - Reciprocal Normalization for Domain Adaptation [31.293016830229313]
Batch normalization (BN) is widely used in modern deep neural networks.
We propose a novel normalization method, Reciprocal Normalization (RN)
RN is more suitable for UDA problems and can be easily integrated into popular domain adaptation methods.
arXiv Detail & Related papers (2021-12-20T12:17:22Z) - Dual-Flow Transformation Network for Deformable Image Registration with
Region Consistency Constraint [95.30864269428808]
Current deep learning (DL)-based image registration approaches learn the spatial transformation from one image to another by leveraging a convolutional neural network.
We present a novel dual-flow transformation network with region consistency constraint which maximizes the similarity of ROIs within a pair of images.
Experiments on four public 3D MRI datasets show that the proposed method achieves the best registration performance in accuracy and generalization.
arXiv Detail & Related papers (2021-12-04T05:30:44Z) - Global and Local Alignment Networks for Unpaired Image-to-Image
Translation [170.08142745705575]
The goal of unpaired image-to-image translation is to produce an output image reflecting the target domain's style.
Due to the lack of attention to the content change in existing methods, semantic information from source images suffers from degradation during translation.
We introduce a novel approach, Global and Local Alignment Networks (GLA-Net)
Our method effectively generates sharper and more realistic images than existing approaches.
arXiv Detail & Related papers (2021-11-19T18:01:54Z) - Rotaflip: A New CNN Layer for Regularization and Rotational Invariance
in Medical Images [0.0]
We propose a CNN layer that performs regularization by applying random rotations of reflections to a small percentage of feature maps after every convolutional layer.
We prove how this concept is beneficial for images with orientational symmetries, such as in medical images.
arXiv Detail & Related papers (2021-08-05T16:13:36Z) - BaMBNet: A Blur-aware Multi-branch Network for Defocus Deblurring [74.34263243089688]
convolutional neural networks (CNNs) have been introduced to the defocus deblurring problem and achieved significant progress.
This study designs a novel blur-aware multi-branch network (BaMBNet) in which different regions (with different blur amounts) should be treated differentially.
Both quantitative and qualitative experiments demonstrate that our BaMBNet outperforms the state-of-the-art methods.
arXiv Detail & Related papers (2021-05-31T07:55:30Z) - DualNorm-UNet: Incorporating Global and Local Statistics for Robust
Medical Image Segmentation [29.368070780337415]
Batch Normalization (BN) is one of the key components for accelerating network training, and has been widely adopted in the medical image analysis field.
We propose to incorporate the semantic class information into normalization layers, so that the activations corresponding to different regions can be modulated differently.
Our approach exploits semantic knowledge at normalization and yields more discriminative features for robust segmentation results.
arXiv Detail & Related papers (2021-03-29T18:09:56Z) - Cross-Scale Internal Graph Neural Network for Image Super-Resolution [147.77050877373674]
Non-local self-similarity in natural images has been well studied as an effective prior in image restoration.
For single image super-resolution (SISR), most existing deep non-local methods only exploit similar patches within the same scale of the low-resolution (LR) input image.
This is achieved using a novel cross-scale internal graph neural network (IGNN)
arXiv Detail & Related papers (2020-06-30T10:48:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.