Towards Harmful Erotic Content Detection through Coreference-Driven Contextual Analysis

• URL: http://arxiv.org/abs/2310.14325v1
• Date: Sun, 22 Oct 2023 15:19:04 GMT
• Title: Towards Harmful Erotic Content Detection through Coreference-Driven Contextual Analysis
• Authors: Inez Okulska and Emilia Wi\'snios
• Abstract summary: This paper introduces a hybrid neural and rule-based context-aware system to identify harmful contextual cues in erotic content. Our model, tested on Polish text, demonstrates a promising accuracy of 84% and a recall of 80%.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adult content detection still poses a great challenge for automation. Existing classifiers primarily focus on distinguishing between erotic and non-erotic texts. However, they often need more nuance in assessing the potential harm. Unfortunately, the content of this nature falls beyond the reach of generative models due to its potentially harmful nature. Ethical restrictions prohibit large language models (LLMs) from analyzing and classifying harmful erotics, let alone generating them to create synthetic datasets for other neural models. In such instances where data is scarce and challenging, a thorough analysis of the structure of such texts rather than a large model may offer a viable solution. Especially given that harmful erotic narratives, despite appearing similar to harmless ones, usually reveal their harmful nature first through contextual information hidden in the non-sexual parts of the narrative. This paper introduces a hybrid neural and rule-based context-aware system that leverages coreference resolution to identify harmful contextual cues in erotic content. Collaborating with professional moderators, we compiled a dataset and developed a classifier capable of distinguishing harmful from non-harmful erotic content. Our hybrid model, tested on Polish text, demonstrates a promising accuracy of 84% and a recall of 80%. Models based on RoBERTa and Longformer without explicit usage of coreference chains achieved significantly weaker results, underscoring the importance of coreference resolution in detecting such nuanced content as harmful erotics. This approach also offers the potential for enhanced visual explainability, supporting moderators in evaluating predictions and taking necessary actions to address harmful content.

Related papers

• A linguistic analysis of undesirable outcomes in the era of generative AI [4.841442157674423]
  We present a comprehensive simulation framework built upon the chat version of LLama2, focusing on the linguistic aspects of the generated content. Our results show that the model produces less lexical rich content across generations, reducing diversity. We find that autophagy transforms the initial model into a more creative, doubtful and confused one, which might provide inaccurate answers.
  arXiv  Detail & Related papers  (2024-10-16T08:02:48Z)
• Detecting sexually explicit content in the context of the child sexual abuse materials (CSAM): end-to-end classifiers and region-based networks [0.0]
  Child sexual abuse materials (CSAM) pose a significant threat to the safety and well-being of children worldwide. This study presents methods for classifying sexually explicit content, which plays a crucial role in the automated CSAM detection system.
  arXiv  Detail & Related papers  (2024-06-20T09:21:08Z)
• SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models [28.23494821842336]
  Text-to-image models may be tricked into generating not-safe-for-work (NSFW) content. We present SafeGen, a framework to mitigate sexual content generation by text-to-image models.
  arXiv  Detail & Related papers  (2024-04-10T00:26:08Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models [52.25049362267279]
  We present a Chinese Bias Benchmark dataset that consists of over 100K questions jointly constructed by human experts and generative language models. The testing instances in the dataset are automatically derived from 3K+ high-quality templates manually authored with stringent quality control. Extensive experiments demonstrate the effectiveness of the dataset in detecting model bias, with all 10 publicly available Chinese large language models exhibiting strong bias in certain categories.
  arXiv  Detail & Related papers  (2023-06-28T14:14:44Z)
• Mutual Information Alleviates Hallucinations in Abstractive Summarization [73.48162198041884]
  We find a simple criterion under which models are significantly more likely to assign more probability to hallucinated content during generation: high model uncertainty. This finding offers a potential explanation for hallucinations: models default to favoring text with high marginal probability, when uncertain about a continuation. We propose a decoding strategy that switches to optimizing for pointwise mutual information of the source and target token--rather than purely the probability of the target token--when the model exhibits uncertainty.
  arXiv  Detail & Related papers  (2022-10-24T13:30:54Z)
• Toxicity Detection with Generative Prompt-based Inference [3.9741109244650823]
  It is a long-known risk that language models (LMs), once trained on corpus containing undesirable content, have the power to manifest biases and toxicity. In this work, we explore the generative variant of zero-shot prompt-based toxicity detection with comprehensive trials on prompt engineering.
  arXiv  Detail & Related papers  (2022-05-24T22:44:43Z)
• Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection [54.38512834521367]
  We study contrast candidate generation and selection as a model-agnostic post-processing technique. We learn a discriminative correction model by generating alternative candidate summaries. This model is then used to select the best candidate as the final output summary.
  arXiv  Detail & Related papers  (2021-04-19T05:39:24Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)
• Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence [48.765579605145454]
  We propose to explicitly segment target text into fragment units and align them with their data correspondences. The resulting architecture maintains the same expressive power as neural attention models. On both E2E and WebNLG benchmarks, we show the proposed model consistently outperforms its neural attention counterparts.
  arXiv  Detail & Related papers  (2020-05-03T14:28:28Z)
• On Faithfulness and Factuality in Abstractive Summarization [17.261247316769484]
  We analyzed limitations of neural text generation models for abstractive document summarization. We found that these models are highly prone to hallucinate content that is unfaithful to the input document. We show that textual entailment measures better correlate with faithfulness than standard metrics.
  arXiv  Detail & Related papers  (2020-05-02T00:09:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.