Understanding Interpretation Difficulty in Harmful Online Communication: Insights from Cybercrime Communities

• URL: http://arxiv.org/abs/2607.07277v1
• Date: Wed, 08 Jul 2026 11:09:06 GMT
• Title: Understanding Interpretation Difficulty in Harmful Online Communication: Insights from Cybercrime Communities
• Abstract summary: This paper presents an exploratory study of interpretation difficulty in Discord chats related to cybercrime.<n>We construct reference interpretations of purposefully selected difficult messages.<n>We then use them to evaluate human and large language model (LLM) interpretations under different context conditions.
• Score: 1.2124621973070164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Harmful online communication often contains slang, coded terms, abbreviations, and community-specific expressions, which make messages difficult to interpret. This paper presents an exploratory study of interpretation difficulty in Discord chats related to cybercrime. We construct reference interpretations of purposefully selected difficult messages, which were reviewed by an expert. We then use them to evaluate human and large language model (LLM) interpretations under different context conditions. The results show that local context alone is often insufficient for humans, while external knowledge and extended conversational context substantially improve human interpretation. For LLMs, local context also improves interpretation, and the larger model performs better. We further conduct a qualitative error analysis and propose a preliminary classification of factors that make harmful chats difficult to interpret. These findings suggest that harmful-content analysis should treat interpretation as an evidence-integration problem, rather than as message-level classification alone.

Related papers

• Beneath the Surface: Investigating LLMs' Capabilities for Communicating with Subtext [14.367312995215393]
  We systematically study whether language models can use subtext in communicative settings.<n>Our evaluation settings range from writing & interpreting allegories to playing multi-agent and multi-modal games.
  arXiv  Detail & Related papers  (2026-04-07T00:14:33Z)
• Slang Context-based Inference Enhancement via Greedy Search-Guided Chain-of-Thought Prompting [3.2239703628634095]
  This paper investigates the challenges of slang inference using large language models.<n>We present a greedy search-guided chain-of-thought framework for slang interpretation.
  arXiv  Detail & Related papers  (2026-01-15T18:15:11Z)
• EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models [6.145223237741804]
  Large language models (LLMs) are increasingly deployed in real-world communication settings, yet their ability to resolve context-dependent ambiguity remains underexplored.<n>We present EMODIS, a new benchmark for evaluating LLMs' capacity to interpret ambiguous emoji expressions under minimal but contrastive textual contexts.
  arXiv  Detail & Related papers  (2025-11-10T15:24:01Z)
• Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translation [70.43884512651668]
  We formalize Genette's (1987) theory of paratexts from literary and translation studies to introduce the task of paratextual explicitation for machine translation.<n>We construct a dataset of 560 expert-aligned paratexts from four English translations of the classical Chinese short story collection Liaozhai.<n>Our findings demonstrate the potential of paratextual explicitation in advancing machine translation beyond linguistic equivalence.
  arXiv  Detail & Related papers  (2025-09-27T16:27:36Z)
• Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answering [68.3400058037817]
  We introduce TREQA (Translation Evaluation via Question-Answering), a framework that extrinsically evaluates translation quality.<n>We show that TREQA is competitive with and, in some cases, outperforms state-of-the-art neural and LLM-based metrics in ranking alternative paragraph-level translations.
  arXiv  Detail & Related papers  (2025-04-10T09:24:54Z)
• Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgments [24.133419857271505]
  Single gold-standard interpretations rarely exist, challenging conventional assumptions in natural language processing. This work introduces the interpretation modeling (IM) task which involves modeling several interpretations of a sentence's underlying semantics. A first-of-its-kind IM dataset is curated to support experiments and analyses.
  arXiv  Detail & Related papers  (2023-11-27T07:50:55Z)
• SOUL: Towards Sentiment and Opinion Understanding of Language [96.74878032417054]
  We propose a new task called Sentiment and Opinion Understanding of Language (SOUL) SOUL aims to evaluate sentiment understanding through two subtasks: Review (RC) and Justification Generation (JG)
  arXiv  Detail & Related papers  (2023-10-27T06:48:48Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [52.992875653864076]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account.<n>We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed.<n>Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• Did they answer? Subjective acts and intents in conversational discourse [48.63528550837949]
  We present the first discourse dataset with multiple and subjective interpretations of English conversation. We show disagreements are nuanced and require a deeper understanding of the different contextual factors.
  arXiv  Detail & Related papers  (2021-04-09T16:34:19Z)
• Interpretable Deep Learning: Interpretations, Interpretability, Trustworthiness, and Beyond [49.93153180169685]
  We introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused. We elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy. We summarize the existing work in evaluating models' interpretability using "trustworthy" interpretation algorithms.
  arXiv  Detail & Related papers  (2021-03-19T08:40:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.