Challenges for Real-Time Toxicity Detection in Online Games

• URL: http://arxiv.org/abs/2407.04383v1
• Date: Fri, 5 Jul 2024 09:38:58 GMT
• Title: Challenges for Real-Time Toxicity Detection in Online Games
• Authors: Lynnette Hui Xian Ng, Adrian Xuan Wei Lim, Michael Miller Yoder,
• Abstract summary: Toxic behaviour and malicious players can ruin the experience, reduce the player base and potentially harm the success of the game and the studio. This article will give a brief overview of the challenges faced in toxic content detection in terms of text, audio and image processing problems, and behavioural toxicity.
• Score: 1.2289361708127877
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Online multiplayer games like League of Legends, Counter Strike, and Skribbl.io create experiences through community interactions. Providing players with the ability to interact with each other through multiple modes also opens a Pandora box. Toxic behaviour and malicious players can ruin the experience, reduce the player base and potentially harming the success of the game and the studio. This article will give a brief overview of the challenges faced in toxic content detection in terms of text, audio and image processing problems, and behavioural toxicity. It also discusses the current practices in company-directed and user-directed content detection and discuss the values and limitations of automated content detection in the age of artificial intelligence.

Related papers

• Online Moderation in Competitive Action Games: How Intervention Affects Player Behaviors [40.395860809162265]
  This study delves into the under-explored realm of understanding the effects of moderation on player behavior within online gaming. We employ a quasi-experimental design and causal inference techniques to examine the impact of moderation in a real-world industry-scale moderation system. Our findings uncover the dual impact moderation has on reducing disruptive behavior and discouraging disruptive players from participating.
  arXiv  Detail & Related papers  (2024-11-01T22:01:25Z)
• Uncovering the Viral Nature of Toxicity in Competitive Online Video Games [0.4681661603096334]
  We analyze proprietary data from the free-to-play first-person action game Call of Duty: Warzone. All of a player's teammates engaging in toxic speech increases their probability of engaging in similar behavior by 26.1 to 30.3 times the average player's likelihood of engaging in toxic speech.
  arXiv  Detail & Related papers  (2024-10-01T18:07:06Z)
• Comprehensive Assessment of Toxicity in ChatGPT [49.71090497696024]
  We evaluate the toxicity in ChatGPT by utilizing instruction-tuning datasets. prompts in creative writing tasks can be 2x more likely to elicit toxic responses. Certain deliberately toxic prompts, designed in earlier studies, no longer yield harmful responses.
  arXiv  Detail & Related papers  (2023-11-03T14:37:53Z)
• Towards Detecting Contextual Real-Time Toxicity for In-Game Chat [5.371337604556311]
  ToxBuster is a scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2.
  arXiv  Detail & Related papers  (2023-10-20T00:29:57Z)
• An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software [64.367830425115]
  Social media platforms are being increasingly misused to spread toxic content, including hate speech, malicious advertising, and pornography. Despite tremendous efforts in developing and deploying content moderation methods, malicious users can evade moderation by embedding texts into images. We propose a metamorphic testing framework for content moderation software.
  arXiv  Detail & Related papers  (2023-08-18T20:33:06Z)
• ToxBuster: In-game Chat Toxicity Buster with BERT [2.764897610820181]
  ToxBuster is a simple and scalable model trained on a relatively large dataset of 194k lines of game chat from Rainbow Six Siege and For Honor. Compared to the existing state-of-the-art, ToxBuster achieves 82.95% (+7) in precision and 83.56% (+57) in recall.
  arXiv  Detail & Related papers  (2023-05-21T18:53:26Z)
• Analyzing Norm Violations in Live-Stream Chat [49.120561596550395]
  We study the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. Our results show that appropriate contextual information can boost moderation performance by 35%.
  arXiv  Detail & Related papers  (2023-05-18T05:58:27Z)
• In-game Toxic Language Detection: Shared Task and Attention Residuals [1.9218741065333018]
  We describe how the in-game toxic language shared task has been established using the real-world in-game chat data. In addition, we propose and introduce the model/framework for toxic language token tagging (slot filling) from the in-game chat.
  arXiv  Detail & Related papers  (2022-11-11T04:33:45Z)
• Collusion Detection in Team-Based Multiplayer Games [57.153233321515984]
  We propose a system that detects colluding behaviors in team-based multiplayer games. The proposed method analyzes the players' social relationships paired with their in-game behavioral patterns. We then automate the detection using Isolation Forest, an unsupervised learning technique specialized in highlighting outliers.
  arXiv  Detail & Related papers  (2022-03-10T02:37:39Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.