APISR: Anime Production Inspired Real-World Anime Super-Resolution

• URL: http://arxiv.org/abs/2403.01598v2
• Date: Thu, 4 Apr 2024 16:12:51 GMT
• Title: APISR: Anime Production Inspired Real-World Anime Super-Resolution
• Authors: Boyang Wang, Fengyu Yang, Xihang Yu, Chao Zhang, Hanbin Zhao,
• Abstract summary: We argue that video networks and datasets are not necessary for anime SR due to the repetition use of hand-drawing frames. Instead, we propose an anime image collection pipeline by choosing the least compressed and the most informative frames from the video sources. We evaluate our method through extensive experiments on the public benchmark, showing our method outperforms state-of-the-art anime dataset-trained approaches.
• Score: 15.501488335115269
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While real-world anime super-resolution (SR) has gained increasing attention in the SR community, existing methods still adopt techniques from the photorealistic domain. In this paper, we analyze the anime production workflow and rethink how to use characteristics of it for the sake of the real-world anime SR. First, we argue that video networks and datasets are not necessary for anime SR due to the repetition use of hand-drawing frames. Instead, we propose an anime image collection pipeline by choosing the least compressed and the most informative frames from the video sources. Based on this pipeline, we introduce the Anime Production-oriented Image (API) dataset. In addition, we identify two anime-specific challenges of distorted and faint hand-drawn lines and unwanted color artifacts. We address the first issue by introducing a prediction-oriented compression module in the image degradation model and a pseudo-ground truth preparation with enhanced hand-drawn lines. In addition, we introduce the balanced twin perceptual loss combining both anime and photorealistic high-level features to mitigate unwanted color artifacts and increase visual clarity. We evaluate our method through extensive experiments on the public benchmark, showing our method outperforms state-of-the-art anime dataset-trained approaches.

Related papers

• NOVA-3D: Non-overlapped Views for 3D Anime Character Reconstruction [14.509202872426942]
  Non-Overlapped Views for 3D textbfAnime Character Reconstruction (NOVA-3D) New framework implements a method for view-aware feature fusion to learn 3D-consistent features effectively. Experiments demonstrate superior reconstruction of anime characters with exceptional detail fidelity.
  arXiv  Detail & Related papers  (2024-05-21T05:31:03Z)
• Diffutoon: High-Resolution Editable Toon Shading via Diffusion Models [25.903156244291168]
  Toon shading is a type of non-photorealistic rendering task of animation. Diffutoon is capable of rendering remarkably detailed, high-resolution, and extended-duration videos in anime style.
  arXiv  Detail & Related papers  (2024-01-29T15:21:37Z)
• Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation [75.91455714614966]
  We propose Scenimefy, a novel semi-supervised image-to-image translation framework. Our approach guides the learning with structure-consistent pseudo paired data. A patch-wise contrastive style loss is introduced to improve stylization and fine details.
  arXiv  Detail & Related papers  (2023-08-24T17:59:50Z)
• AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models [24.94532405404846]
  We propose a novel method called AnimeDiffusion using diffusion models that performs anime face line drawing colorization automatically. We conduct an anime face line drawing colorization benchmark dataset, which contains 31696 training data and 579 testing data. We demonstrate AnimeDiffusion outperforms state-of-the-art GANs-based models for anime face drawing colorization.
  arXiv  Detail & Related papers  (2023-03-20T14:15:23Z)
• Learning 3D Photography Videos via Self-supervised Diffusion on Single Images [105.81348348510551]
  3D photography renders a static image into a video with appealing 3D visual effects. Existing approaches typically first conduct monocular depth estimation, then render the input frame to subsequent frames with various viewpoints. We present a novel task: out-animation, which extends the space and time of input objects.
  arXiv  Detail & Related papers  (2023-02-21T16:18:40Z)
• AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies [98.65469430034246]
  Existing datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements. We present a new 2D animation visual correspondence dataset, AnimeRun, by converting open source 3D movies to full scenes in 2D style. Our analyses show that the proposed dataset not only resembles real anime more in image composition, but also possesses richer and more complex motion patterns compared to existing datasets.
  arXiv  Detail & Related papers  (2022-11-10T17:26:21Z)
• AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation [84.52819242283852]
  We propose a novel framework to translate a portrait photo-face into an anime appearance. Our aim is to synthesize anime-faces which are style-consistent with a given reference anime-face. Existing methods often fail to transfer the styles of reference anime-faces, or introduce noticeable artifacts/distortions in the local shapes of their generated faces.
  arXiv  Detail & Related papers  (2021-02-24T22:47:38Z)
• Exploiting Raw Images for Real-Scene Super-Resolution [105.18021110372133]
  We study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. We also develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images.
  arXiv  Detail & Related papers  (2021-02-02T16:10:15Z)
• Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement [78.58603635621591]
  Training an unpaired synthetic-to-real translation network in image space is severely under-constrained. We propose a semi-supervised approach that operates on the disentangled shading and albedo layers of the image. Our two-stage pipeline first learns to predict accurate shading in a supervised fashion using physically-based renderings as targets.
  arXiv  Detail & Related papers  (2020-03-27T21:45:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.