When Harmful Content Gets Camouflaged: Unveiling Perception Failure of LVLMs with CamHarmTI

• URL: http://arxiv.org/abs/2512.03087v1
• Date: Sat, 29 Nov 2025 06:39:47 GMT
• Title: When Harmful Content Gets Camouflaged: Unveiling Perception Failure of LVLMs with CamHarmTI
• Abstract summary: Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial.<n>This paper introduces CamHarmTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions.
• Score: 32.60708757460534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial, such as online content moderation. However, real-world harmful content is often camouflaged, relying on nuanced text-image interplay, such as memes or images with embedded malicious text, to evade detection. This raises a key question: \textbf{can LVLMs perceive such camouflaged harmful content as sensitively as humans do?} In this paper, we introduce CamHarmTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions. CamHarmTI consists of over 4,500 samples across three types of image-text posts. Experiments on 100 human users and 12 mainstream LVLMs reveal a clear perceptual gap: humans easily recognize such content (e.g., over 95.75\% accuracy), whereas current LVLMs often fail (e.g., ChatGPT-4o achieves only 2.10\% accuracy). Moreover, fine-tuning experiments demonstrate that \bench serves as an effective resource for improving model perception, increasing accuracy by 55.94\% for Qwen2.5VL-7B. Attention analysis and layer-wise probing further reveal that fine-tuning enhances sensitivity primarily in the early layers of the vision encoder, promoting a more integrated scene understanding. These findings highlight the inherent perceptual limitations in LVLMs and offer insight into more human-aligned visual reasoning systems.

Related papers

• What the Eyes See, the LLMs Miss: Exploiting Human Perception for Adversarial Text Attacks [9.745253975157912]
  Large language model (LLM)-powered content moderation systems have become a critical defense against harmful online content.<n>We show that content that is readily recognized as harmful by humans can become effectively invisible to automated moderation systems.<n>We introduce a class of Human-Perceptible Adversarial Attacks (HPAA), in which harmful expressions are embedded into otherwise benign text.
  arXiv  Detail & Related papers  (2026-06-08T16:21:34Z)
• FineViT: Progressively Unlocking Fine-Grained Perception with Dense Recaptions [52.366937743884314]
  FineViT is a novel vision encoder specifically designed to unlock fine-grained perception.<n>We introduce FineViT, a novel vision encoder specifically designed to unlock fine-grained perception.
  arXiv  Detail & Related papers  (2026-03-18T03:39:04Z)
• Evaluating the Sensitivity of LLMs to Harmful Contents in Long Input [53.19281984086319]
  Large language models (LLMs) increasingly support applications that rely on extended context, from document processing to retrieval-augmented generation.<n>We evaluate LLMs' sensitivity to harmful content under extended context, varying type (explicit vs. implicit), position (beginning, middle, end), prevalence (0.01-0.50 of the prompt), and context length (600-6000 tokens).<n>We observe similar patterns: performance peaks at moderate harmful prevalence (0.25) but declines when content is very sparse or dominant; recall decreases with increasing context length; harmful sentences at the beginning are generally detected more reliably; and explicit content is more consistently recognized than implicit
  arXiv  Detail & Related papers  (2025-10-07T12:33:21Z)
• Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens [0.0]
  Large vision-language models (LVLMs) have demonstrated remarkable multimodal comprehension and reasoning capabilities.<n>LVLMs tend to over-rely on textual prompts and internal knowledge of large language models, generating descriptions inconsistent with visual cues.<n>We propose a training-free method to mitigate object hallucination.
  arXiv  Detail & Related papers  (2025-08-04T13:40:59Z)
• SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking [31.356290235986332]
  Vision-language models (VLMs) excel in semantic tasks but falter at a core human capability.<n>We introduce HC-Bench, a benchmark of 112 images with hidden text, objects, and illusions.<n>We propose SemVink (Semantic Visual Thinking), which unlocks >99% accuracy by eliminating redundant visual noise.
  arXiv  Detail & Related papers  (2025-06-03T12:33:47Z)
• Blind Spot Navigation: Evolutionary Discovery of Sensitive Semantic Concepts for LVLMs [24.76767896607915]
  Recent research suggests that models may be particularly sensitive to certain semantics in visual inputs, making them prone to errors.<n>Inspired by this, in this paper we conducted the first exploration on large vision-language models (LVLMs)<n>We found that LVLMs indeed are susceptible to hallucinations and various errors when facing specific semantic concepts in images.
  arXiv  Detail & Related papers  (2025-05-21T08:45:43Z)
• Mitigating Hallucination for Large Vision Language Model by Inter-Modality Correlation Calibration Decoding [66.06337890279839]
  Large vision-language models (LVLMs) have shown remarkable capabilities in visual-language understanding for downstream multi-modal tasks.<n>LVLMs still suffer from generating hallucinations in complex generation tasks, leading to inconsistencies between visual inputs and generated content.<n>We propose an Inter-Modality Correlation Decoding (IMCCD) method to mitigate hallucinations in LVLMs in a training-free manner.
  arXiv  Detail & Related papers  (2025-01-03T17:56:28Z)
• Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning [151.4060202671114]
  multimodal large language models (MLLMs) have shown unprecedented capabilities in advancing vision-language tasks.<n>This paper introduces a novel bottom-up reasoning framework to address hallucinations in MLLMs.<n>Our framework systematically addresses potential issues in both visual and textual inputs by verifying and integrating perception-level information with cognition-level commonsense knowledge.
  arXiv  Detail & Related papers  (2024-12-15T09:10:46Z)
• Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models [30.26685485474035]
  Large Vision-Language Models (LVLMs) have rapidly advanced in recent years.<n>The prevalent issue known as the hallucination' problem has emerged as a significant bottleneck.<n>We propose a simple yet effective method named Self-Introspective Decoding (SID)
  arXiv  Detail & Related papers  (2024-08-04T13:50:17Z)
• MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification [1.3654846342364308]
  We introduce MetaToken, a lightweight binary classifier to detect hallucinations on the token-level at negligible cost.<n>Based on a statistical analysis, we reveal key factors of hallucinations in Large Vision Language Models.<n>We evaluate our method on four state-of-the-art LVLMs demonstrating the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-05-29T15:28:42Z)
• Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs [52.497823009176074]
  Large Vision-Language Models (LVLMs) often produce responses that misalign with factual information, a phenomenon known as hallucinations.<n>We introduce Visual Description Grounded Decoding (VDGD), a training-free method designed to enhance visual perception and improve reasoning capabilities in LVLMs.
  arXiv  Detail & Related papers  (2024-05-24T16:21:59Z)
• Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance [51.30560006045442]
  Image-gRounded guIdaNcE (MARINE) is a framework that is both training-free and API-free.<n>MARINE effectively and efficiently reduces object hallucinations during inference by introducing image-grounded guidance to LVLMs.<n>Our framework's flexibility further allows for the integration of multiple vision models, enabling more reliable and robust object-level guidance.
  arXiv  Detail & Related papers  (2024-02-13T18:59:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.