Masked Face Inpainting Through Residual Attention UNet

• URL: http://arxiv.org/abs/2209.08850v1
• Date: Mon, 19 Sep 2022 08:49:53 GMT
• Title: Masked Face Inpainting Through Residual Attention UNet
• Authors: Md Imran Hosen and Md Baharul Islam
• Abstract summary: This paper proposes a blind mask face inpainting method using residual attention UNet. A residual block feeds info to the next layer and directly into the layers about two hops away to solve the vanishing gradient problem. Experiments on the publicly available CelebA dataset show the feasibility and robustness of our proposed model.
• Score: 0.7868449549351486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Realistic image restoration with high texture areas such as removing face masks is challenging. The state-of-the-art deep learning-based methods fail to guarantee high-fidelity, cause training instability due to vanishing gradient problems (e.g., weights are updated slightly in initial layers) and spatial information loss. They also depend on intermediary stage such as segmentation meaning require external mask. This paper proposes a blind mask face inpainting method using residual attention UNet to remove the face mask and restore the face with fine details while minimizing the gap with the ground truth face structure. A residual block feeds info to the next layer and directly into the layers about two hops away to solve the gradient vanishing problem. Besides, the attention unit helps the model focus on the relevant mask region, reducing resources and making the model faster. Extensive experiments on the publicly available CelebA dataset show the feasibility and robustness of our proposed model. Code is available at \url{https://github.com/mdhosen/Mask-Face-Inpainting-Using-Residual-Attention-Unet}

Related papers

• Face Mask Removal with Region-attentive Face Inpainting [0.7433327915285965]
  We propose a generative face inpainting method to recover/reconstruct the masked part of a face. Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss. We synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks.
  arXiv  Detail & Related papers  (2024-09-10T20:10:11Z)
• CLR-Face: Conditional Latent Refinement for Blind Face Restoration Using Score-Based Diffusion Models [57.9771859175664]
  Recent generative-prior-based methods have shown promising blind face restoration performance. Generating fine-grained facial details faithful to inputs remains a challenging problem. We introduce a diffusion-based-prior inside a VQGAN architecture that focuses on learning the distribution over uncorrupted latent embeddings.
  arXiv  Detail & Related papers  (2024-02-08T23:51:49Z)
• Learning Representations for Masked Facial Recovery [8.124282476398843]
  pandemic of these recent years has led to a dramatic increase in people wearing protective masks in public venues. One way to address the problem is to revert to face recovery methods as a preprocessing step. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask.
  arXiv  Detail & Related papers  (2022-12-28T22:22:15Z)
• Towards Improved Input Masking for Convolutional Neural Networks [66.99060157800403]
  We propose a new masking method for CNNs we call layer masking. We show that our method is able to eliminate or minimize the influence of the mask shape or color on the output of the model. We also demonstrate how the shape of the mask may leak information about the class, thus affecting estimates of model reliance on class-relevant features.
  arXiv  Detail & Related papers  (2022-11-26T19:31:49Z)
• Shape-Aware Masking for Inpainting in Medical Imaging [49.61617087640379]
  Inpainting has been proposed as a successful deep learning technique for unsupervised medical image model discovery. We introduce a method for generating shape-aware masks for inpainting, which aims at learning the statistical shape prior. We propose an unsupervised guided masking approach based on an off-the-shelf inpainting model and a superpixel over-segmentation algorithm.
  arXiv  Detail & Related papers  (2022-07-12T18:35:17Z)
• Layered Depth Refinement with Mask Guidance [61.10654666344419]
  We formulate a novel problem of mask-guided depth refinement that utilizes a generic mask to refine the depth prediction of SIDE models. Our framework performs layered refinement and inpainting/outpainting, decomposing the depth map into two separate layers signified by the mask and the inverse mask. We empirically show that our method is robust to different types of masks and initial depth predictions, accurately refining depth values in inner and outer mask boundary regions.
  arXiv  Detail & Related papers  (2022-06-07T06:42:44Z)
• A Unified Framework for Masked and Mask-Free Face Recognition via Feature Rectification [19.417191498842044]
  We propose a unified framework, named Face Feature Rectification Network (FFR-Net), for recognizing both masked and mask-free faces alike. We introduce rectification blocks to rectify features extracted by a state-of-the-art recognition model, in both spatial and channel dimensions. Experiments show that our framework can learn a rectified feature space for recognizing both masked and mask-free faces effectively.
  arXiv  Detail & Related papers  (2022-02-15T12:37:59Z)
• RePaint: Inpainting using Denoising Diffusion Probabilistic Models [161.74792336127345]
  Free-form inpainting is the task of adding new content to an image in the regions specified by an arbitrary binary mask. We propose RePaint: A Denoising Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks. We validate our method for both faces and general-purpose image inpainting using standard and extreme masks.
  arXiv  Detail & Related papers  (2022-01-24T18:40:15Z)
• Segmentation-Reconstruction-Guided Facial Image De-occlusion [48.952656891182826]
  Occlusions are very common in face images in the wild, leading to the degraded performance of face-related tasks. This paper proposes a novel face de-occlusion model based on face segmentation and 3D face reconstruction.
  arXiv  Detail & Related papers  (2021-12-15T10:40:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.