Can Language Model Moderators Improve the Health of Online Discourse?

• URL: http://arxiv.org/abs/2311.10781v2
• Date: Mon, 6 May 2024 17:44:12 GMT
• Title: Can Language Model Moderators Improve the Health of Online Discourse?
• Authors: Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May,
• Abstract summary: We establish a systematic definition of conversational moderation effectiveness grounded on moderation literature. We propose a comprehensive evaluation framework to assess models' moderation capabilities independently of human intervention.
• Score: 26.191337231826246
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational moderation of online communities is crucial to maintaining civility for a constructive environment, but it is challenging to scale and harmful to moderators. The inclusion of sophisticated natural language generation modules as a force multiplier to aid human moderators is a tantalizing prospect, but adequate evaluation approaches have so far been elusive. In this paper, we establish a systematic definition of conversational moderation effectiveness grounded on moderation literature and establish design criteria for conducting realistic yet safe evaluation. We then propose a comprehensive evaluation framework to assess models' moderation capabilities independently of human intervention. With our framework, we conduct the first known study of language models as conversational moderators, finding that appropriately prompted models that incorporate insights from social science can provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation.

Related papers

• Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning [84.94709351266557]
  We focus on the trustworthiness of language models with respect to retrieval augmentation. We deem that retrieval-augmented language models have the inherent capabilities of supplying response according to both contextual and parametric knowledge. Inspired by aligning language models with human preference, we take the first step towards aligning retrieval-augmented language models to a status where it responds relying merely on the external evidence.
  arXiv  Detail & Related papers  (2024-10-22T09:25:21Z)
• ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models [53.00812898384698]
  We argue that human evaluation of generative large language models (LLMs) should be a multidisciplinary undertaking. We highlight how cognitive biases can conflate fluent information and truthfulness, and how cognitive uncertainty affects the reliability of rating scores such as Likert. We propose the ConSiDERS-The-Human evaluation framework consisting of 6 pillars -- Consistency, Scoring Criteria, Differentiating, User Experience, Responsible, and Scalability.
  arXiv  Detail & Related papers  (2024-05-28T22:45:28Z)
• The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content [2.2618341648062477]
  This paper examines the role of intent in content moderation systems. We review state of the art detection models and benchmark training datasets for online abuse to assess their awareness and ability to capture intent.
  arXiv  Detail & Related papers  (2024-05-17T18:05:13Z)
• Recourse for reclamation: Chatting with generative language models [2.877217169371665]
  We extend the concept of algorithmic recourse to generative language models. We provide users a novel mechanism to achieve their desired prediction by dynamically setting thresholds for toxicity filtering. A pilot study supports the potential of our proposed recourse mechanism.
  arXiv  Detail & Related papers  (2024-03-21T15:14:25Z)
• Controllable Mixed-Initiative Dialogue Generation through Prompting [50.03458333265885]
  Mixed-initiative dialogue tasks involve repeated exchanges of information and conversational control. Agents gain control by generating responses that follow particular dialogue intents or strategies, prescribed by a policy planner. Standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents. We instead prompt large language models as a drop-in replacement to fine-tuning on conditional generation.
  arXiv  Detail & Related papers  (2023-05-06T23:11:25Z)
• Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation [68.9440575276396]
  This survey aims to provide an overview of the recent research that has leveraged human feedback to improve natural language generation. First, we introduce an encompassing formalization of feedback, and identify and organize existing research into a taxonomy following this formalization. Second, we discuss how feedback can be described by its format and objective, and cover the two approaches proposed to use feedback (either for training or decoding): directly using the feedback or training feedback models. Third, we provide an overview of the nascent field of AI feedback, which exploits large language models to make judgments based on a set of principles and minimize the need for
  arXiv  Detail & Related papers  (2023-05-01T17:36:06Z)
• Dialogue Evaluation with Offline Reinforcement Learning [2.580163308334609]
  Task-oriented dialogue systems aim to fulfill user goals through natural language interactions. They are ideally evaluated with human users, which is unattainable to do at every iteration of the development phase. We propose the use of offline reinforcement learning for dialogue evaluation based on a static corpus.
  arXiv  Detail & Related papers  (2022-09-02T08:32:52Z)
• Democratizing Ethical Assessment of Natural Language Generation Models [0.0]
  Natural language generation models are computer systems that generate coherent language when prompted with a sequence of words as context. Despite their ubiquity and many beneficial applications, language generation models also have the potential to inflict social harms. Ethical assessment of these models is therefore critical. This article introduces a new tool to democratize and standardize ethical assessment of natural language generation models.
  arXiv  Detail & Related papers  (2022-06-30T12:20:31Z)
• Knowledge-Grounded Dialogue Generation with Pre-trained Language Models [74.09352261943911]
  We study knowledge-grounded dialogue generation with pre-trained language models. We propose equipping response generation defined by a pre-trained language model with a knowledge selection module.
  arXiv  Detail & Related papers  (2020-10-17T16:49:43Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.