Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study

• URL: http://arxiv.org/abs/2505.21710v1
• Date: Tue, 27 May 2025 19:53:12 GMT
• Title: Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study
• Authors: Barbarestani Baran, Maks Isa, Vossen Piek,
• Abstract summary: This study evaluates the effectiveness of ChatGPT, an advanced AI model for natural language processing, in identifying targeting and inappropriate language in online comments.<n>We compared ChatGPT's performance against crowd-sourced annotations and expert evaluations to assess its accuracy, scope of detection, and consistency.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study evaluates the effectiveness of ChatGPT, an advanced AI model for natural language processing, in identifying targeting and inappropriate language in online comments. With the increasing challenge of moderating vast volumes of user-generated content on social network sites, the role of AI in content moderation has gained prominence. We compared ChatGPT's performance against crowd-sourced annotations and expert evaluations to assess its accuracy, scope of detection, and consistency. Our findings highlight that ChatGPT performs well in detecting inappropriate content, showing notable improvements in accuracy through iterative refinements, particularly in Version 6. However, its performance in targeting language detection showed variability, with higher false positive rates compared to expert judgments. This study contributes to the field by demonstrating the potential of AI models like ChatGPT to enhance automated content moderation systems while also identifying areas for further improvement. The results underscore the importance of continuous model refinement and contextual understanding to better support automated moderation and mitigate harmful online behavior.

Related papers

• Improving Harmful Text Detection with Joint Retrieval and External Knowledge [16.68620974551506]
  This study proposes a joint retrieval framework that integrates pre-trained language models with knowledge graphs to improve the accuracy and robustness of harmful text detection.<n> Experimental results demonstrate that the joint retrieval approach significantly outperforms single-model baselines.
  arXiv  Detail & Related papers  (2025-04-03T06:37:55Z)
• $C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction [80.57232374640911]
  We propose a model-agnostic strategy called the Mask-And-Recover (MAR)<n>MAR integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules.<n>To better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model.
  arXiv  Detail & Related papers  (2025-04-01T13:01:30Z)
• Using ChatGPT to Score Essays and Short-Form Constructed Responses [0.0]
  Investigation focused on various prediction models, including linear regression, random forest, gradient boost, and boost. ChatGPT's performance was evaluated against human raters using quadratic weighted kappa (QWK) metrics. Study concludes that ChatGPT can complement human scoring but requires additional development to be reliable for high-stakes assessments.
  arXiv  Detail & Related papers  (2024-08-18T16:51:28Z)
• Toward Practical Automatic Speech Recognition and Post-Processing: a Call for Explainable Error Benchmark Guideline [12.197453599489963]
  We propose the development of an Error Explainable Benchmark (EEB) dataset. This dataset, while considering both speech- and text-level, enables a granular understanding of the model's shortcomings. Our proposition provides a structured pathway for a more real-world-centric' evaluation, allowing for the detection and rectification of nuanced system weaknesses.
  arXiv  Detail & Related papers  (2024-01-26T03:42:45Z)
• Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark [4.970614891967042]
  We analyze GPT's spatial reasoning performance on the StepGame benchmark. We identify proficiency in mapping natural language text to spatial relations but limitations in multi-hop reasoning. We deploy Chain-of-thought and Tree-of-thoughts prompting strategies, offering insights into GPT's cognitive process"
  arXiv  Detail & Related papers  (2024-01-08T16:13:08Z)
• Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement [97.41630939425731]
  We propose a transformative system that combines the conversational intelligence of ChatGPT with multiple IRE methods. Our case studies demonstrate that Clarity ChatGPT effectively improves the generalization and interaction capabilities in the IRE.
  arXiv  Detail & Related papers  (2023-11-20T11:51:13Z)
• DEMASQ: Unmasking the ChatGPT Wordsmith [63.8746084667206]
  We propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. Our method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machine-generated content and (ii) the alterations made by humans to evade previous detection methods.
  arXiv  Detail & Related papers  (2023-11-08T21:13:05Z)
• Leveraging Large Language Models for Automated Dialogue Analysis [12.116834890063146]
  This paper investigates the ability of a state-of-the-art large language model (LLM), ChatGPT-3.5, to perform dialogue behavior detection for nine categories in real human-bot dialogues. Our findings reveal that neither specialized models nor ChatGPT have yet achieved satisfactory results for this task, falling short of human performance.
  arXiv  Detail & Related papers  (2023-09-12T18:03:55Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Consistency Analysis of ChatGPT [65.268245109828]
  This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions.
  arXiv  Detail & Related papers  (2023-03-11T01:19:01Z)
• On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective [67.98821225810204]
  We evaluate the robustness of ChatGPT from the adversarial and out-of-distribution perspective. Results show consistent advantages on most adversarial and OOD classification and translation tasks. ChatGPT shows astounding performance in understanding dialogue-related texts.
  arXiv  Detail & Related papers  (2023-02-22T11:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.