SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detection

• URL: http://arxiv.org/abs/2609.13794v1
• Date: Sat, 12 Sep 2026 08:12:24 GMT
• Title: SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detection
• Abstract summary: SyRHM is a framework that decomposes harmful meme detection into meaning-grounded retrieval and multi-stage reasoning.<n>SyRHM retrieves semantically related memes by parsing multimodal content into textual elements and descriptions.<n> Experiments on FHM, HarM, and MultiOff demonstrate the effectiveness of SyRHM.
• Score: 9.138331410334581
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting harmful memes is critical for maintaining safe online communities. However, harmful intent is often implicit, arising from visual-textual incongruity and cultural stereotypes, which challenges existing multimodal detectors. We propose SyRHM, a framework that decomposes harmful meme detection into meaning-grounded retrieval and symbolic-language-enhanced multi-stage reasoning. SyRHM retrieves semantically related memes by parsing multimodal content into textual elements and descriptions, providing grounded context beyond surface-level similarity. Building on the retrieved context, SyRHM uses a translator stage to convert multimodal inputs into symbolic intermediate representations, and then performs multi-stage reasoning via planner and solver stages, enabling expressive and interpretable analysis of harmful intent. Experiments on FHM, HarM, and MultiOff demonstrate the effectiveness of SyRHM, achieving superior performance on most evaluation settings against multimodal and reasoning-based baselines, while providing reasoning traces for harmful content. The code is available at: https://github.com/Scabbards1500/SyRHM

Related papers

• Buried in Textual Debt: Context Pruning with Visual Evidence Preservation for MLLM Agents [16.82888862043366]
  We propose a Kullback-Leibler-guided framework for pruning accumulated reasoning in tool-use agents.<n> SPARE achieves the highest average accuracy among pruning methods while removing 37.89-64.58% of reasoning tokens.<n>This favorable accuracy-context trade-off shows that reducing textual dominance restores reliance on visual evidence and mitigates over-conditioning on self-generated language.
  arXiv  Detail & Related papers  (2026-08-24T08:29:17Z)
• Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation [53.844308305341166]
  We propose textbfAgentic ASR, a closed-loop framework that combines a single-pass ASR front-end with semantic correction, intent routing, and reasoning-based editing.<n> Experiments on multilingual, named-entity-intensive, and code-switching benchmarks show that iterative interaction consistently reduces semantic errors.
  arXiv  Detail & Related papers  (2026-05-28T06:23:31Z)
• More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection [49.07221734365168]
  Hate speech on social media is critical for securing cyberspace, yet relies heavily on automated detection systems.<n>We characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion.<n>We propose the Asymmetric Reasoning via Courtroom Agent DEbate framework to effectively decipher these complex cues.
  arXiv  Detail & Related papers  (2026-03-22T15:46:23Z)
• Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models [54.10540442330978]
  Existing benchmarks are typically multilingual but text-only, or multimodal but monolingual.<n>Recent multilingual red-teaming efforts render harmful prompts into images, yet rely heavily on typography-style visuals.<n>We introduce a benchmark of 100,440 harmful image-text pairs across 10 languages, explicitly partitioned into image-dominant and text-dominant subsets.
  arXiv  Detail & Related papers  (2026-01-30T09:18:13Z)
• METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark [48.78602579128459]
  We introduce METER, a unified benchmark for interpretable forgery detection spanning images, videos, audio, and audio-visual content.<n>Our dataset comprises four tracks, each requiring not only real-vs-fake classification but also evidence-chain-based explanations.
  arXiv  Detail & Related papers  (2025-07-22T03:42:51Z)
• Detecting Harmful Memes with Decoupled Understanding and Guided CoT Reasoning [26.546646866501735]
  We introduce U-CoT+, a novel framework for harmful meme detection.<n>We first develop a high-fidelity meme-to-text pipeline that converts visual memes into detail-preserving textual descriptions.<n>This design decouples meme interpretation from meme classification, thus avoiding immediate reasoning over complex raw visual content.
  arXiv  Detail & Related papers  (2025-06-10T06:10:45Z)
• Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answering [0.5587293092389789]
  Hateful memes often evade traditional text-only or image-only detection systems, particularly when they employ subtle or coded references.<n>We propose a multimodal hate detection framework that integrates OCR to extract embedded text, captioning to describe visual content neutrally, sub-label classification for granular categorization, RAG for contextually relevant retrieval, and VQA for iterative analysis of symbolic and contextual cues.<n> Experimental results on the Facebook Hateful Memes dataset reveal that the proposed framework exceeds the performance of unimodal and conventional multimodal models in both accuracy and AUC-ROC.
  arXiv  Detail & Related papers  (2025-04-23T13:52:14Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• HateSieve: A Contrastive Learning Framework for Detecting and Segmenting Hateful Content in Multimodal Memes [8.97062933976566]
  textscHateSieve is a framework designed to enhance the detection and segmentation of hateful elements in memes.<n>textscHateSieve features a novel Contrastive Meme Generator that creates semantically paired memes.<n> Empirical experiments on the Hateful Meme show that textscHateSieve not only surpasses existing LMMs in performance with fewer trainable parameters but also offers a robust mechanism for precisely identifying and isolating hateful content.
  arXiv  Detail & Related papers  (2024-08-11T14:56:06Z)
• Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights [50.89022445197919]
  We propose a speech-specific risk taxonomy, covering 8 risk categories under hostility (malicious sarcasm and threats), malicious imitation (age, gender, ethnicity), and stereotypical biases (age, gender, ethnicity) Based on the taxonomy, we create a small-scale dataset for evaluating current LMMs capability in detecting these categories of risk.
  arXiv  Detail & Related papers  (2024-06-25T10:08:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.