KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection

• URL: http://arxiv.org/abs/2601.21796v1
• Date: Thu, 29 Jan 2026 14:41:36 GMT
• Title: KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection
• Authors: Yaocong Li, Leihan Zhang, Le Zhang, Qiang Yan,
• Abstract summary: We propose a Knowledge-Injected Dual-Head Learning framework for knowledge-grounded harmful meme detection.<n> KID adopts a label-constrained distillation paradigm to decompose complex meme understanding into structured reasoning chains that explicitly link visual evidence, background knowledge, and classification labels.
• Score: 6.006318485752551
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Internet memes have become pervasive carriers of digital culture on social platforms. However, their heavy reliance on metaphors and sociocultural context also makes them subtle vehicles for harmful content, posing significant challenges for automated content moderation. Existing approaches primarily focus on intra-modal and inter-modal signal analysis, while the understanding of implicit toxicity often depends on background knowledge that is not explicitly present in the meme itself. To address this challenge, we propose KID, a Knowledge-Injected Dual-Head Learning framework for knowledge-grounded harmful meme detection. KID adopts a label-constrained distillation paradigm to decompose complex meme understanding into structured reasoning chains that explicitly link visual evidence, background knowledge, and classification labels. These chains guide the learning process by grounding external knowledge in meme-specific contexts. In addition, KID employs a dual-head architecture that jointly optimizes semantic generation and classification objectives, enabling aligned linguistic reasoning while maintaining stable decision boundaries. Extensive experiments on five multilingual datasets spanning English, Chinese, and low-resource Bengali demonstrate that KID achieves SOTA performance on both binary and multi-label harmful meme detection tasks, improving over previous best methods by 2.1%--19.7% across primary evaluation metrics. Ablation studies further confirm the effectiveness of knowledge injection and dual-head joint learning, highlighting their complementary contributions to robust and generalizable meme understanding. The code and data are available at https://github.com/PotatoDog1669/KID.

Related papers

• Towards LLM-Empowered Knowledge Tracing via LLM-Student Hierarchical Behavior Alignment in Hyperbolic Space [24.868649493405528]
  Knowledge Tracing (KT) diagnoses students' concept mastery through continuous learning state monitoring in education.<n>Existing methods rely on ID-based sequences or shallow textual features.<n>This paper proposes a Large Language Model Hyperbolic Aligned Knowledge Tracing framework.
  arXiv  Detail & Related papers  (2026-02-26T11:17:31Z)
• CKDA: Cross-modality Knowledge Disentanglement and Alignment for Visible-Infrared Lifelong Person Re-identification [77.07028925223383]
  Lifelong person Re-IDentification aims to match the same person employing continuously collected individual data from different scenarios.<n>To achieve continuous all-day person matching across day and night, Visible-Infrared Lifelong person Re-IDentification (VI-LReID) focuses on sequential training on data from visible and infrared modalities.<n>Existing methods typically exploit cross-modal knowledge distillation to alleviate the catastrophic forgetting of old knowledge.
  arXiv  Detail & Related papers  (2025-11-19T01:30:29Z)
• Hindsight Distillation Reasoning with Knowledge Encouragement Preference for Knowledge-based Visual Question Answering [55.368681418311894]
  Existing Knowledge-based Visual Question Answering (KBVQA) methods either utilize implicit knowledge in multimodal large language models (MLLMs) via in-context learning or explicit knowledge via retrieval augmented generation.<n>We provide a Hindsight Distilled Reasoning (HinD) framework with Knowledge Encouragement Preference Optimization (KEPO)<n> Experiments on OK-VQA and A-OKVQA validate the effectiveness of HinD, showing that HinD with elicited reasoning from 7B-size MLLM achieves superior performance without commercial model APIs or outside knowledge.
  arXiv  Detail & Related papers  (2025-11-14T10:03:23Z)
• MemeMind: A Large-Scale Multimodal Dataset with Chain-of-Thought Reasoning for Harmful Meme Detection [4.09109557328609]
  Harmful memes pose significant challenges for automated detection due to implicit semantics and complex multimodal interactions.<n>MemeMind is a novel dataset featuring scientifically rigorous standards, large scale, diversity, bilingual support (Chinese and English), and detailed Chain-of-Thought (CoT) annotations.<n>We propose an innovative detection framework, MemeGuard, which effectively integrates multimodal information with reasoning process modeling.
  arXiv  Detail & Related papers  (2025-06-15T13:45:30Z)
• KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning [74.21524111840652]
  This paper proposes textbfKaLM, a textitKnowledge-aligned Language Modeling approach.<n>It fine-tunes autoregressive large language models to align with KG knowledge via the joint objective of explicit knowledge alignment and implicit knowledge alignment.<n> Notably, our method achieves a significant performance boost in evaluations of knowledge-driven tasks.
  arXiv  Detail & Related papers  (2024-12-06T11:08:24Z)
• Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes [8.42736066868944]
  We propose a novel framework that integrates Knowledge Distillation (KD) from Large Visual Language Models (LVLMs) and knowledge infusion to enhance the performance of toxicity detection in hateful memes.<n>Our approach extracts sub-knowledge graphs from ConceptNet, a large-scale commonsense Knowledge Graph (KG) to be infused within a compact VLM framework.<n> Experimental results from our study on two hate speech benchmark datasets demonstrate superior performance over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-11-19T02:39:28Z)
• What Really is Commonsense Knowledge? [58.5342212738895]
  We survey existing definitions of commonsense knowledge, ground into the three frameworks for defining concepts, and consolidate them into a unified definition of commonsense knowledge. We then use the consolidated definition for annotations and experiments on the CommonsenseQA and CommonsenseQA 2.0 datasets. Our study shows that there exists a large portion of non-commonsense-knowledge instances in the two datasets, and a large performance gap on these two subsets.
  arXiv  Detail & Related papers  (2024-11-06T14:54:19Z)
• Towards Automated Knowledge Integration From Human-Interpretable Representations [55.2480439325792]
  We introduce and motivate theoretically the principles of informed meta-learning enabling automated and controllable inductive bias selection.<n>We empirically demonstrate the potential benefits and limitations of informed meta-learning in improving data efficiency and generalisation.
  arXiv  Detail & Related papers  (2024-02-25T15:08:37Z)
• CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning [45.62134354858683]
  CANDLE is a framework that iteratively performs conceptualization and instantiation over commonsense knowledge bases. By applying CANDLE to ATOMIC, we construct a comprehensive knowledge base comprising six million conceptualizations and instantiated commonsense knowledge triples.
  arXiv  Detail & Related papers  (2024-01-14T13:24:30Z)
• Dual Semantic Knowledge Composed Multimodal Dialog Systems [114.52730430047589]
  We propose a novel multimodal task-oriented dialog system named MDS-S2. It acquires the context related attribute and relation knowledge from the knowledge base. We also devise a set of latent query variables to distill the semantic information from the composed response representation.
  arXiv  Detail & Related papers  (2023-05-17T06:33:26Z)
• Unsupervised Domain Adaptation on Person Re-Identification via Dual-level Asymmetric Mutual Learning [108.86940401125649]
  This paper proposes a Dual-level Asymmetric Mutual Learning method (DAML) to learn discriminative representations from a broader knowledge scope with diverse embedding spaces. The knowledge transfer between two networks is based on an asymmetric mutual learning manner. Experiments in Market-1501, CUHK-SYSU, and MSMT17 public datasets verified the superiority of DAML over state-of-the-arts.
  arXiv  Detail & Related papers  (2023-01-29T12:36:17Z)
• Distilling Holistic Knowledge with Graph Neural Networks [37.86539695906857]
  Knowledge Distillation (KD) aims at transferring knowledge from a larger well-optimized teacher network to a smaller learnable student network. Existing KD methods have mainly considered two types of knowledge, namely the individual knowledge and the relational knowledge. We propose to distill the novel holistic knowledge based on an attributed graph constructed among instances.
  arXiv  Detail & Related papers  (2021-08-12T02:47:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.