Unsupervised Manga Character Re-identification via Face-body and
Spatial-temporal Associated Clustering
- URL: http://arxiv.org/abs/2204.04621v1
- Date: Sun, 10 Apr 2022 07:28:41 GMT
- Title: Unsupervised Manga Character Re-identification via Face-body and
Spatial-temporal Associated Clustering
- Authors: Zhimin Zhang, Zheng Wang, Wei Hu
- Abstract summary: The artistic expression and stylistic limitations of manga pose many challenges to the re-identification problem.
Inspired by the idea that some content-related features may help clustering, we propose a Face-body and Spatial-temporal Associated Clustering method.
In the face-body combination module, a face-body graph is constructed to solve problems such as exaggeration and deformation in artistic creation.
In the spatial-temporal relationship correction module, we analyze the appearance features of characters and design a temporal-spatial-related triplet loss to fine-tune the clustering.
- Score: 21.696847342192072
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In the past few years, there has been a dramatic growth in e-manga
(electronic Japanese-style comics). Faced with the booming demand for manga
research and the large amount of unlabeled manga data, we raised a new task,
called unsupervised manga character re-identification. However, the artistic
expression and stylistic limitations of manga pose many challenges to the
re-identification problem. Inspired by the idea that some content-related
features may help clustering, we propose a Face-body and Spatial-temporal
Associated Clustering method (FSAC). In the face-body combination module, a
face-body graph is constructed to solve problems such as exaggeration and
deformation in artistic creation by using the integrity of the image. In the
spatial-temporal relationship correction module, we analyze the appearance
features of characters and design a temporal-spatial-related triplet loss to
fine-tune the clustering. Extensive experiments on a manga book dataset with
109 volumes validate the superiority of our method in unsupervised manga
character re-identification.
Related papers
- Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models [26.010509997863196]
We propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga.
Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.
arXiv Detail & Related papers (2024-03-13T05:33:52Z) - The Manga Whisperer: Automatically Generating Transcriptions for Comics [55.544015596503726]
We present a unified model, Magi, that is able to detect panels, text boxes and character boxes.
We propose a novel approach that is able to sort the detected text boxes in their reading order and generate a dialogue transcript.
arXiv Detail & Related papers (2024-01-18T18:59:09Z) - inkn'hue: Enhancing Manga Colorization from Multiple Priors with
Alignment Multi-Encoder VAE [0.0]
We propose a specialized framework for manga colorization.
We leverage established models for shading and vibrant coloring using a multi-encoder VAE.
This structured workflow ensures clear and colorful results, with the option to incorporate reference images and manual hints.
arXiv Detail & Related papers (2023-11-03T09:33:32Z) - Scenimefy: Learning to Craft Anime Scene via Semi-Supervised
Image-to-Image Translation [75.91455714614966]
We propose Scenimefy, a novel semi-supervised image-to-image translation framework.
Our approach guides the learning with structure-consistent pseudo paired data.
A patch-wise contrastive style loss is introduced to improve stylization and fine details.
arXiv Detail & Related papers (2023-08-24T17:59:50Z) - Dense Multitask Learning to Reconfigure Comics [63.367664789203936]
We develop a MultiTask Learning (MTL) model to achieve dense predictions for comics panels.
Our method can successfully identify the semantic units as well as the notion of 3D in comic panels.
arXiv Detail & Related papers (2023-07-16T15:10:34Z) - MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with
Informative-Preserved Reconstruction and Self-Distilled Consistency [120.9499803967496]
We propose a novel informative-preserved reconstruction, which explores local statistics to discover and preserve the representative structured points.
Our method can concentrate on modeling regional geometry and enjoy less ambiguity for masked reconstruction.
By combining informative-preserved reconstruction on masked areas and consistency self-distillation from unmasked areas, a unified framework called MM-3DScene is yielded.
arXiv Detail & Related papers (2022-12-20T01:53:40Z) - Skeleton-free Pose Transfer for Stylized 3D Characters [53.33996932633865]
We present the first method that automatically transfers poses between stylized 3D characters without skeletal rigging.
We propose a novel pose transfer network that predicts the character skinning weights and deformation transformations jointly to articulate the target character to match the desired pose.
Our method is trained in a semi-supervised manner absorbing all existing character data with paired/unpaired poses and stylized shapes.
arXiv Detail & Related papers (2022-07-28T20:05:57Z) - AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised
Anime Face Generation [84.52819242283852]
We propose a novel framework to translate a portrait photo-face into an anime appearance.
Our aim is to synthesize anime-faces which are style-consistent with a given reference anime-face.
Existing methods often fail to transfer the styles of reference anime-faces, or introduce noticeable artifacts/distortions in the local shapes of their generated faces.
arXiv Detail & Related papers (2021-02-24T22:47:38Z) - MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology
of Manga Drawing [27.99490750445691]
We propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation.
Inspired by how experienced manga artists draw manga, MangaGAN generates the geometric features of manga face by a designed GAN model.
To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module.
arXiv Detail & Related papers (2020-04-22T15:23:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.