WSSL: Weighted Self-supervised Learning Framework For Image-inpainting

• URL: http://arxiv.org/abs/2211.13856v2
• Date: Thu, 24 Aug 2023 19:28:08 GMT
• Title: WSSL: Weighted Self-supervised Learning Framework For Image-inpainting
• Authors: Shubham Gupta, Rahul Kunigal Ravishankar, Madhoolika Gangaraju, Poojasree Dwarkanath and Natarajan Subramanyam
• Abstract summary: Image inpainting is a process of regenerating lost parts of the image. Supervised algorithm-based methods have shown excellent results but have two significant drawbacks. We propose a novel self-supervised learning framework for image-inpainting: Weighted Self-Supervised Learning.
• Score: 18.297463645457693
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image inpainting is the process of regenerating lost parts of the image. Supervised algorithm-based methods have shown excellent results but have two significant drawbacks. They do not perform well when tested with unseen data. They fail to capture the global context of the image, resulting in a visually unappealing result. We propose a novel self-supervised learning framework for image-inpainting: Weighted Self-Supervised Learning (WSSL) to tackle these problems. We designed WSSL to learn features from multiple weighted pretext tasks. These features are then utilized for the downstream task, image-inpainting. To improve the performance of our framework and produce more visually appealing images, we also present a novel loss function for image inpainting. The loss function takes advantage of both reconstruction loss and perceptual loss functions to regenerate the image. Our experimentation shows WSSL outperforms previous methods, and our loss function helps produce better results.

Related papers

• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
  We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
  arXiv  Detail & Related papers  (2023-08-29T07:11:52Z)
• Rethinking PRL: A Multiscale Progressively Residual Learning Network for Inverse Halftoning [3.632876183725243]
  inverse halftoning is a classic image restoration task, aiming to recover continuous-tone images from halftone images with only bilevel pixels. We propose an end-to-end multiscale progressively residual learning network (MSPRL), which has a UNet architecture and takes multiscale input images.
  arXiv  Detail & Related papers  (2023-05-27T03:37:33Z)
• Unlocking Masked Autoencoders as Loss Function for Image and Video Restoration [19.561055022474786]
  We study the potential of loss and raise our belief learned loss function empowers the learning capability of neural networks for image and video restoration'' We investigate the efficacy of our belief from three perspectives: 1) from task-customized MAE to native MAE, 2) from image task to video task, and 3) from transformer structure to convolution neural network structure.
  arXiv  Detail & Related papers  (2023-03-29T02:41:08Z)
• Learning Prior Feature and Attention Enhanced Image Inpainting [63.21231753407192]
  This paper incorporates the pre-training based Masked AutoEncoder (MAE) into the inpainting model. We propose to use attention priors from MAE to make the inpainting model learn more long-distance dependencies between masked and unmasked regions.
  arXiv  Detail & Related papers  (2022-08-03T04:32:53Z)
• Training a Better Loss Function for Image Restoration [17.20936270604533]
  We show that a single natural image is sufficient to train a lightweight feature extractor that outperforms state-of-the-art loss functions in single image super resolution. We propose a novel Multi-Scale Discriminative Feature (MDF) loss comprising a series of discriminators, trained to penalize errors introduced by a generator.
  arXiv  Detail & Related papers  (2021-03-26T17:29:57Z)
• Image Restoration by Deep Projected GSURE [115.57142046076164]
  Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution. We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
  arXiv  Detail & Related papers  (2021-02-04T08:52:46Z)
• Why Do Better Loss Functions Lead to Less Transferable Features? [93.47297944685114]
  This paper studies how the choice of training objective affects the transferability of the hidden representations of convolutional neural networks trained on ImageNet. We show that many objectives lead to statistically significant improvements in ImageNet accuracy over vanilla softmax cross-entropy, but the resulting fixed feature extractors transfer substantially worse to downstream tasks.
  arXiv  Detail & Related papers  (2020-10-30T17:50:31Z)
• Improving Text to Image Generation using Mode-seeking Function [5.92166950884028]
  We develop a specialized mode-seeking loss function for the generation of distinct images. We validate our model on the Caltech Birds dataset and the Microsoft COCO dataset. Experimental results demonstrate that our model works very well compared to some state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-08-19T12:58:32Z)
• Powers of layers for image-to-image translation [60.5529622990682]
  We propose a simple architecture to address unpaired image-to-image translation tasks. We start from an image autoencoder architecture with fixed weights. For each task we learn a residual block operating in the latent space, which is iteratively called until the target domain is reached.
  arXiv  Detail & Related papers  (2020-08-13T09:02:17Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.