Reinforcement Learning for Efficient Toxicity Detection in Competitive Online Video Games

• URL: http://arxiv.org/abs/2503.20968v1
• Date: Wed, 26 Mar 2025 20:13:30 GMT
• Title: Reinforcement Learning for Efficient Toxicity Detection in Competitive Online Video Games
• Authors: Jacob Morrier, Rafal Kocielnik, R. Michael Alvarez,
• Abstract summary: This article considers the problem of efficient sampling for toxicity detection in competitive online video games.<n>We propose a contextual bandit algorithm that makes monitoring decisions based on variables associated with toxic behavior.<n>Using data from the popular first-person action game Call of Duty: Modern Warfare III, we show that our algorithm consistently outperforms baseline algorithms.
• Score: 1.9201314880477047
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online platforms take proactive measures to detect and address undesirable behavior, aiming to focus these resource-intensive efforts where such behavior is most prevalent. This article considers the problem of efficient sampling for toxicity detection in competitive online video games. To make optimal monitoring decisions, video game service operators need estimates of the likelihood of toxic behavior. If no model is available for these predictions, one must be estimated in real time. To close this gap, we propose a contextual bandit algorithm that makes monitoring decisions based on a small set of variables that, according to domain expertise, are associated with toxic behavior. This algorithm balances exploration and exploitation to optimize long-term outcomes and is deliberately designed for easy deployment in production. Using data from the popular first-person action game Call of Duty: Modern Warfare III, we show that our algorithm consistently outperforms baseline algorithms that rely solely on players' past behavior. This finding has substantive implications for the nature of toxicity. It also illustrates how domain expertise can be harnessed to help video game service operators identify and mitigate toxicity, ultimately fostering a safer and more enjoyable gaming experience.

Related papers

• Who's Gaming the System? A Causally-Motivated Approach for Detecting Strategic Adaptation [12.528928000871405]
  We consider a multi-agent setting where the goal is to identify the "worst offenders:" agents that are gaming most aggressively.<n>We introduce a framework in which each agent's tendency to game is parameterized via a scalar.<n>By recasting the problem as a causal effect estimation problem where different agents represent different "treatments," we prove that a ranking of all agents by their gaming parameters is identifiable.
  arXiv  Detail & Related papers  (2024-12-02T22:07:48Z)
• Uncovering the Viral Nature of Toxicity in Competitive Online Video Games [0.4681661603096334]
  We analyze proprietary data from the free-to-play first-person action game Call of Duty: Warzone.<n>All of a player's teammates engaging in toxic speech increases their probability of engaging in similar behavior by 26.1 to 30.3 times the average player's likelihood of engaging in toxic speech.
  arXiv  Detail & Related papers  (2024-10-01T18:07:06Z)
• Challenges for Real-Time Toxicity Detection in Online Games [1.2289361708127877]
  Toxic behaviour and malicious players can ruin the experience, reduce the player base and potentially harm the success of the game and the studio. This article will give a brief overview of the challenges faced in toxic content detection in terms of text, audio and image processing problems, and behavioural toxicity.
  arXiv  Detail & Related papers  (2024-07-05T09:38:58Z)
• The Update-Equivalence Framework for Decision-Time Planning [78.44953498421854]
  We introduce an alternative framework for decision-time planning that is not based on solving subgames, but rather on update equivalence. We derive a provably sound search algorithm for fully cooperative games based on mirror descent and a search algorithm for adversarial games based on magnetic mirror descent.
  arXiv  Detail & Related papers  (2023-04-25T20:28:55Z)
• Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks [83.28949556413717]
  We study the problem of computing an approximate Nash equilibrium of continuous-action game without access to gradients. We model players' strategies using artificial neural networks. This paper is the first to solve general continuous-action games with unrestricted mixed strategies and without any gradient information.
  arXiv  Detail & Related papers  (2022-11-29T05:16:41Z)
• Modeling Content Creator Incentives on Algorithm-Curated Platforms [76.53541575455978]
  We study how algorithmic choices affect the existence and character of (Nash) equilibria in exposure games. We propose tools for numerically finding equilibria in exposure games, and illustrate results of an audit on the MovieLens and LastFM datasets.
  arXiv  Detail & Related papers  (2022-06-27T08:16:59Z)
• Collusion Detection in Team-Based Multiplayer Games [57.153233321515984]
  We propose a system that detects colluding behaviors in team-based multiplayer games. The proposed method analyzes the players' social relationships paired with their in-game behavioral patterns. We then automate the detection using Isolation Forest, an unsupervised learning technique specialized in highlighting outliers.
  arXiv  Detail & Related papers  (2022-03-10T02:37:39Z)
• Discovering Imperfectly Observable Adversarial Actions using Anomaly Detection [0.24244694855867271]
  Anomaly detection is a method for discovering unusual and suspicious behavior. We propose two algorithms for solving such games. Experiments show that both algorithms are applicable for cases with low feature space dimensions.
  arXiv  Detail & Related papers  (2020-04-22T15:31:53Z)
• Approximate exploitability: Learning a best response in large games [31.066412349285994]
  We introduce ISMCTS-BR, a scalable search-based deep reinforcement learning algorithm for learning a best response to an agent. We demonstrate the technique in several two-player zero-sum games against a variety of agents.
  arXiv  Detail & Related papers  (2020-04-20T23:36:40Z)
• Efficient exploration of zero-sum stochastic games [83.28949556413717]
  We investigate the increasingly important and common game-solving setting where we do not have an explicit description of the game but only oracle access to it through gameplay. During a limited-duration learning phase, the algorithm can control the actions of both players in order to try to learn the game and how to play it well. Our motivation is to quickly learn strategies that have low exploitability in situations where evaluating the payoffs of a queried strategy profile is costly.
  arXiv  Detail & Related papers  (2020-02-24T20:30:38Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.