A Fuzzy Evaluation of Sentence Encoders on Grooming Risk Classification

• URL: http://arxiv.org/abs/2502.12576v1
• Date: Tue, 18 Feb 2025 06:26:46 GMT
• Title: A Fuzzy Evaluation of Sentence Encoders on Grooming Risk Classification
• Authors: Geetanjali Bihani, Julia Rayz,
• Abstract summary: Children are becoming increasingly vulnerable to the risk of grooming in online settings. Predators evade detection using indirect and coded language. Previous studies have fine-tuned Transformers to automatically identify grooming in chat conversations.
• Score: 5.616884466478886
• License:
• Abstract: With the advent of social media, children are becoming increasingly vulnerable to the risk of grooming in online settings. Detecting grooming instances in an online conversation poses a significant challenge as the interactions are not necessarily sexually explicit, since the predators take time to build trust and a relationship with their victim. Moreover, predators evade detection using indirect and coded language. While previous studies have fine-tuned Transformers to automatically identify grooming in chat conversations, they overlook the impact of coded and indirect language on model predictions, and how these align with human perceptions of grooming. In this paper, we address this gap and evaluate bi-encoders on the task of classifying different degrees of grooming risk in chat contexts, for three different participant groups, i.e. law enforcement officers, real victims, and decoys. Using a fuzzy-theoretic framework, we map human assessments of grooming behaviors to estimate the actual degree of grooming risk. Our analysis reveals that fine-tuned models fail to tag instances where the predator uses indirect speech pathways and coded language to evade detection. Further, we find that such instances are characterized by a higher presence of out-of-vocabulary (OOV) words in samples, causing the model to misclassify. Our findings highlight the need for more robust models to identify coded language from noisy chat inputs in grooming contexts.

Related papers

• Evaluating Language Models on Grooming Risk Estimation Using Fuzzy Theory [4.997673761305337]
  In this paper, we study whether SBERT can effectively discern varying degrees of grooming risk inherent in conversations. Our analysis reveals that while fine-tuning aids language models in learning to assign grooming scores, they show high variance in predictions. These errors appear in cases that 1) utilize indirect speech pathways to manipulate victims and 2) lack sexually explicit content.
  arXiv  Detail & Related papers  (2025-02-18T05:59:54Z)
• Diagnosing Hate Speech Classification: Where Do Humans and Machines Disagree, and Why? [0.0]
  We use cosine similarity ratio, embedding regression, and manual re-annotation to diagnose hate speech classification. Female annotators are more sensitive to racial slurs that target the black population. In diagnosing where machines disagree with human annotators, we found that machines make fewer mistakes than humans.
  arXiv  Detail & Related papers  (2024-10-14T04:39:45Z)
• Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals [91.59906995214209]
  We propose a new evaluation method, Counterfactual Attentiveness Test (CAT) CAT uses counterfactuals by replacing part of the input with its counterpart from a different example, expecting an attentive model to change its prediction. We show that GPT3 becomes less attentive with an increased number of demonstrations, while its accuracy on the test data improves.
  arXiv  Detail & Related papers  (2023-11-16T06:27:35Z)
• Fine-tuning Language Models for Factuality [96.5203774943198]
  Large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations' In this work, we fine-tune language models to be more factual, without human labeling.
  arXiv  Detail & Related papers  (2023-11-14T18:59:15Z)
• Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and Abusive Texts [2.406214748890827]
  This paper proposes an approach to detection of online sexual predatory chats and abusive language using the open-source pretrained Llama 2 7B- parameter model. We fine-tune the LLM using datasets with different sizes, imbalance degrees, and languages (i.e., English, Roman Urdu and Urdu) Experimental results show a strong performance of the proposed approach, which performs proficiently and consistently across three distinct datasets.
  arXiv  Detail & Related papers  (2023-08-28T16:18:50Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis [69.07674653828565]
  Machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis. We propose a family of regularization methods, NFL (doN't Forget your Language) to mitigate spurious correlations in text classification.
  arXiv  Detail & Related papers  (2023-05-23T03:55:50Z)
• Hate Speech and Offensive Language Detection using an Emotion-aware Shared Encoder [1.8734449181723825]
  Existing works on hate speech and offensive language detection produce promising results based on pre-trained transformer models. This paper addresses a multi-task joint learning approach which combines external emotional features extracted from another corpora. Our findings demonstrate that emotional knowledge helps to more reliably identify hate speech and offensive language across datasets.
  arXiv  Detail & Related papers  (2023-02-17T09:31:06Z)
• Can We Automate the Analysis of Online Child Sexual Exploitation Discourse? [18.20420363291303]
  Social media's growing popularity raises concerns around children's online safety. Research into online sexual grooming has often relied on domain experts to manually annotate conversations. We test how well-automated methods can detect conversational behaviors and replace an expert human annotator.
  arXiv  Detail & Related papers  (2022-09-25T21:18:50Z)
• On the Robustness of Language Encoders against Grammatical Errors [66.05648604987479]
  We collect real grammatical errors from non-native speakers and conduct adversarial attacks to simulate these errors on clean text data. Results confirm that the performance of all tested models is affected but the degree of impact varies.
  arXiv  Detail & Related papers  (2020-05-12T11:01:44Z)
• Learning an Unreferenced Metric for Online Dialogue Evaluation [53.38078951628143]
  We propose an unreferenced automated evaluation metric that uses large pre-trained language models to extract latent representations of utterances. We show that our model achieves higher correlation with human annotations in an online setting, while not requiring true responses for comparison during inference.
  arXiv  Detail & Related papers  (2020-05-01T20:01:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.