Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification

• URL: http://arxiv.org/abs/2308.09096v1
• Date: Thu, 17 Aug 2023 16:48:41 GMT
• Title: Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification
• Authors: G\"urkan Soykan, Deniz Yuret, Tevfik Metin Sezgin
• Abstract summary: We introduce a robust framework that combines metric learning with a novel 'Identity-Aware' self-supervision method. Our approach involves processing both facial and bodily features within a unified network architecture. By extensively validating our method using in-series and inter-series evaluation metrics, we demonstrate its effectiveness in consistently re-identifying comic characters.
• Score: 2.4624325014867763
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Character re-identification, recognizing characters consistently across different panels in comics, presents significant challenges due to limited annotated data and complex variations in character appearances. To tackle this issue, we introduce a robust semi-supervised framework that combines metric learning with a novel 'Identity-Aware' self-supervision method by contrastive learning of face and body pairs of characters. Our approach involves processing both facial and bodily features within a unified network architecture, facilitating the extraction of identity-aligned character embeddings that capture individual identities while preserving the effectiveness of face and body features. This integrated character representation enhances feature extraction and improves character re-identification compared to re-identification by face or body independently, offering a parameter-efficient solution. By extensively validating our method using in-series and inter-series evaluation metrics, we demonstrate its effectiveness in consistently re-identifying comic characters. Compared to existing methods, our approach not only addresses the challenge of character re-identification but also serves as a foundation for downstream tasks since it can produce character embeddings without restrictions of face and body availability, enriching the comprehension of comic books. In our experiments, we leverage two newly curated datasets: the 'Comic Character Instances Dataset', comprising over a million character instances and the 'Comic Sequence Identity Dataset', containing annotations of identities within more than 3000 sets of four consecutive comic panels that we collected.

Related papers

• DIFFER: Disentangling Identity Features via Semantic Cues for Clothes-Changing Person Re-ID [15.204652332980672]
  Clothes-changing person re-identification (CC-ReID) aims to recognize individuals under different clothing scenarios. Current CC-ReID approaches either concentrate on modeling body shape using additional modalities including silhouette, pose, and body mesh. We propose DIFFER: Disentangle Identity Features From Entangled Representations, a novel adversarial learning method that leverages textual descriptions to disentangle identity features.
  arXiv  Detail & Related papers  (2025-03-28T23:40:59Z)
• See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification [16.845045499676793]
  Cloth-changing person re-identification (CC-ReID) aims to match individuals across multiple surveillance cameras despite variations in clothing. Existing methods typically focus on mitigating the effects of clothing changes or enhancing ID-relevant features. We propose a novel prompt learning framework, Semantic Contextual Integration (SCI), for CC-ReID.
  arXiv  Detail & Related papers  (2024-12-02T10:11:16Z)
• Foundation Cures Personalization: Recovering Facial Personalized Models' Prompt Consistency [33.35678923549471]
  FreeCure is a training-free framework that harnesses the intrinsic knowledge from the foundation models themselves to improve the prompt consistency of personalization models. We enhance multiple attributes in the outputs of personalization models through a novel noise-blending strategy and an inversion-based process.
  arXiv  Detail & Related papers  (2024-11-22T15:21:38Z)
• Disentangled Representations for Short-Term and Long-Term Person Re-Identification [33.76874948187976]
  We propose a new generative adversarial network, dubbed identity shuffle GAN (IS-GAN) It disentangles identity-related and unrelated features from person images through an identity-shuffling technique. Experimental results validate the effectiveness of IS-GAN, showing state-of-the-art performance on standard reID benchmarks.
  arXiv  Detail & Related papers  (2024-09-09T02:09:49Z)
• Content and Salient Semantics Collaboration for Cloth-Changing Person Re-Identification [74.10897798660314]
  Cloth-changing person Re-IDentification aims at recognizing the same person with clothing changes across non-overlapping cameras. We propose the Content and Salient Semantics Collaboration framework, facilitating cross-parallel semantics interaction and refinement. Our framework is simple yet effective, and the vital design is the Semantics Mining and Refinement (SMR) module.
  arXiv  Detail & Related papers  (2024-05-26T15:17:28Z)
• Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Models [79.21968152209193]
  We introduce the NewEpisode benchmark to evaluate generative models' adaptability in generating new stories with fresh characters. We propose EpicEvo, a method that customizes a diffusion-based visual story generation model with a single story featuring the new characters seamlessly integrating them into established character dynamics.
  arXiv  Detail & Related papers  (2024-05-20T07:54:03Z)
• Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm [31.06269858216316]
  We propose Infinite-ID, an ID-semantics decoupling paradigm for identity-preserved personalization. We introduce an identity-enhanced training, incorporating an additional image cross-attention module to capture sufficient ID information. We also introduce a feature interaction mechanism that combines a mixed attention module with an AdaIN-mean operation to seamlessly merge the two streams.
  arXiv  Detail & Related papers  (2024-03-18T13:39:53Z)
• StableIdentity: Inserting Anybody into Anywhere at First Sight [57.99693188913382]
  We propose StableIdentity, which allows identity-consistent recontextualization with just one face image. We are the first to directly inject the identity learned from a single image into video/3D generation without finetuning.
  arXiv  Detail & Related papers  (2024-01-29T09:06:15Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF) This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module. Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• StyleID: Identity Disentanglement for Anonymizing Faces [4.048444203617942]
  The main contribution of the paper is the design of a feature-preserving anonymization framework, StyleID. As part of the contribution, we present a novel disentanglement metric, three complementing disentanglement methods, and new insights into identity disentanglement. StyleID provides tunable privacy, has low computational complexity, and is shown to outperform current state-of-the-art solutions.
  arXiv  Detail & Related papers  (2022-12-28T12:04:24Z)
• Disentangling Identity and Pose for Facial Expression Recognition [54.50747989860957]
  We propose an identity and pose disentangled facial expression recognition (IPD-FER) model to learn more discriminative feature representation. For identity encoder, a well pre-trained face recognition model is utilized and fixed during training, which alleviates the restriction on specific expression training data. By comparing the difference between synthesized neutral and expressional images of the same individual, the expression component is further disentangled from identity and pose.
  arXiv  Detail & Related papers  (2022-08-17T06:48:13Z)
• Protecting Celebrities with Identity Consistency Transformer [119.67996461810304]
  Identity Consistency Transformer focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions. We show that Identity Consistency Transformer exhibits superior generalization ability not only across different datasets but also across various types of image degradation forms found in real-world applications including deepfake videos.
  arXiv  Detail & Related papers  (2022-03-02T18:59:58Z)
• Semantic Consistency and Identity Mapping Multi-Component Generative Adversarial Network for Person Re-Identification [39.605062525247135]
  We propose a semantic consistency and identity mapping multi-component generative adversarial network (SC-IMGAN) which provides style adaptation from one to many domains. Our proposed method outperforms state-of-the-art techniques on six challenging person Re-ID datasets.
  arXiv  Detail & Related papers  (2021-04-28T14:12:29Z)
• Separating Content from Style Using Adversarial Learning for Recognizing Text in the Wild [103.51604161298512]
  We propose an adversarial learning framework for the generation and recognition of multiple characters in an image. Our framework can be integrated into recent recognition methods to achieve new state-of-the-art recognition accuracy.
  arXiv  Detail & Related papers  (2020-01-13T12:41:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.