CHASM: Unveiling Covert Advertisements on Chinese Social Media

• URL: http://arxiv.org/abs/2604.20511v1
• Date: Wed, 22 Apr 2026 12:53:36 GMT
• Title: CHASM: Unveiling Covert Advertisements on Chinese Social Media
• Abstract summary: We present the CHASM, a first-of-its-kind dataset designed to evaluate the capability of Multimodal Large Language Models (MLLMs) in detecting covert advertisements on social media.<n> CHASM is a high-quality, anonymized, manually curated dataset consisting of 4,992 instances, based on real-world scenarios from the Chinese social media platform Rednote.<n>The results show that under both zero-shot and in-context learning settings, none of the current MLLMs are sufficiently reliable for detecting covert advertisements.
• Score: 14.328587848128201
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current benchmarks for evaluating large language models (LLMs) in social media moderation completely overlook a serious threat: covert advertisements, which disguise themselves as regular posts to deceive and mislead consumers into making purchases, leading to significant ethical and legal concerns. In this paper, we present the CHASM, a first-of-its-kind dataset designed to evaluate the capability of Multimodal Large Language Models (MLLMs) in detecting covert advertisements on social media. CHASM is a high-quality, anonymized, manually curated dataset consisting of 4,992 instances, based on real-world scenarios from the Chinese social media platform Rednote. The dataset was collected and annotated under strict privacy protection and quality control protocols. It includes many product experience sharing posts that closely resemble covert advertisements, making the dataset particularly challenging.The results show that under both zero-shot and in-context learning settings, none of the current MLLMs are sufficiently reliable for detecting covert advertisements.Our further experiments revealed that fine-tuning open-source MLLMs on our dataset yielded noticeable performance gains. However, significant challenges persist, such as detecting subtle cues in comments and differences in visual and textual structures.We provide in-depth error analysis and outline future research directions. We hope our study can serve as a call for the research community and platform moderators to develop more precise defenses against this emerging threat.

Related papers

• Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions [16.02370545528885]
  Large Language Models (LLMs) have emerged as powerful tools that impact information integrity on social media platforms.<n>This comprehensive review examines the dual role of LLMs in both facilitating and mitigating various information integrity challenges.
  arXiv  Detail & Related papers  (2026-08-05T02:32:43Z)
• Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges [55.425628316813174]
  Multi-modal Large Language Models (MLLMs) process both text and images.<n>Compared to text-only models, MLLMs can extract and expose sensitive information embedded in images.<n>Some MLLMs are susceptible to privacy breaches, leaking sensitive data embedded in images or stored in memory.
  arXiv  Detail & Related papers  (2026-06-08T07:19:42Z)
• ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction [14.012874564599272]
  ZoFia is a novel two-stage zero-shot fake news detection framework.<n>First, we introduce Hierarchical Salience to quantify the importance of entities in the news content.<n>We then propose the SC-MMR algorithm to effectively select an informative and diverse set of keywords.
  arXiv  Detail & Related papers  (2025-11-03T03:29:42Z)
• Profiling News Media for Factuality and Bias Using LLMs and the Fact-Checking Methodology of Human Experts [29.95198868148809]
  We propose a novel methodology that emulates the criteria that professional fact-checkers use to assess the factuality and political bias of an entire outlet.<n>We provide an in-depth error analysis of the effect of media popularity and region on model performance.
  arXiv  Detail & Related papers  (2025-06-14T15:49:20Z)
• Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions [23.024212585005714]
  Large Language Models (LLMs) have gained enormous attention in recent years due to their capability of understanding and generating natural languages.<n>This paper aims to provide a comprehensive terminology of concepts and methods around this field and facilitate the community.
  arXiv  Detail & Related papers  (2025-06-08T16:20:12Z)
• An exploration of features to improve the generalisability of fake news detection models [0.0]
  Existing NLP and supervised Machine Learning methods perform well under cross-validation but struggle to generalise across datasets.<n>This issue stems from coarsely labelled training data, where articles are labelled based on their publisher.<n>This study demonstrates that meaningful features can still be extracted from coarsely labelled data to improve real-world robustness.
  arXiv  Detail & Related papers  (2025-02-27T17:26:56Z)
• LLM-PBE: Assessing Data Privacy in Large Language Models [111.58198436835036]
  Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs.
  arXiv  Detail & Related papers  (2024-08-23T01:37:29Z)
• MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models [51.19622266249408]
  MultiTrust is the first comprehensive and unified benchmark on the trustworthiness of MLLMs.<n>Our benchmark employs a rigorous evaluation strategy that addresses both multimodal risks and cross-modal impacts.<n>Extensive experiments with 21 modern MLLMs reveal some previously unexplored trustworthiness issues and risks.
  arXiv  Detail & Related papers  (2024-06-11T08:38:13Z)
• A Survey on Detection of LLMs-Generated Content [97.87912800179531]
  The ability to detect LLMs-generated content has become of paramount importance. We aim to provide a detailed overview of existing detection strategies and benchmarks. We also posit the necessity for a multi-faceted approach to defend against various attacks.
  arXiv  Detail & Related papers  (2023-10-24T09:10:26Z)
• Privacy in Large Language Models: Attacks, Defenses and Future Directions [84.73301039987128]
  We analyze the current privacy attacks targeting large language models (LLMs) and categorize them according to the adversary's assumed capabilities. We present a detailed overview of prominent defense strategies that have been developed to counter these privacy attacks.
  arXiv  Detail & Related papers  (2023-10-16T13:23:54Z)
• ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media [74.93847489218008]
  We present a novel task, identifying manipulation of news on social media, which aims to detect manipulation in social media posts and identify manipulated or inserted information.<n>To study this task, we have proposed a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles.<n>Our analysis demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance.
  arXiv  Detail & Related papers  (2023-05-23T16:40:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.