From Moderation to Mediation: Can LLMs Serve as Mediators in Online Flame Wars?

• URL: http://arxiv.org/abs/2512.03005v1
• Date: Tue, 02 Dec 2025 18:31:18 GMT
• Title: From Moderation to Mediation: Can LLMs Serve as Mediators in Online Flame Wars?
• Authors: Dawei Li, Abdullah Alnaibari, Arslan Bisharat, Manny Sandoval, Deborah Hall, Yasin Silva, Huan Liu,
• Abstract summary: Large language models (LLMs) have opened new possibilities for AI for good applications.<n>This work explores whether LLMs can serve as moderators that detect harmful content, but as mediators capable of understanding and de-escalating online conflicts.<n>Our framework decomposes mediation into two subtasks: judgment, where an LLM evaluates the fairness and emotional dynamics of a conversation, and steering, where it generates empathetic, de-escalatory messages.
• Score: 7.926773786209838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid advancement of large language models (LLMs) has opened new possibilities for AI for good applications. As LLMs increasingly mediate online communication, their potential to foster empathy and constructive dialogue becomes an important frontier for responsible AI research. This work explores whether LLMs can serve not only as moderators that detect harmful content, but as mediators capable of understanding and de-escalating online conflicts. Our framework decomposes mediation into two subtasks: judgment, where an LLM evaluates the fairness and emotional dynamics of a conversation, and steering, where it generates empathetic, de-escalatory messages to guide participants toward resolution. To assess mediation quality, we construct a large Reddit-based dataset and propose a multi-stage evaluation pipeline combining principle-based scoring, user simulation, and human comparison. Experiments show that API-based models outperform open-source counterparts in both reasoning and intervention alignment when doing mediation. Our findings highlight both the promise and limitations of current LLMs as emerging agents for online social mediation.

Related papers

• The Collective Turing Test: Large Language Models Can Generate Realistic Multi-User Discussions [0.4605116997238364]
  Large Language Models (LLMs) offer new avenues to simulate online communities and social media.<n>We evaluated whether LLMs can convincingly mimic human group conversations on social media.
  arXiv  Detail & Related papers  (2025-10-29T17:01:20Z)
• DEBATE: A Large-Scale Benchmark for Role-Playing LLM Agents in Multi-Agent, Long-Form Debates [10.609797175227644]
  We introduce DEBATE, the first large-scale empirical benchmark to evaluate the authenticity of the interaction between multi-agent role-playing LLMs.<n>We systematically evaluate and identify critical discrepancies between simulated and authentic group dynamics.
  arXiv  Detail & Related papers  (2025-10-29T02:21:10Z)
• Can LLMs Simulate Social Media Engagement? A Study on Action-Guided Response Generation [51.44040615856536]
  This paper analyzes large language models' ability to simulate social media engagement through action guided response generation.<n>We benchmark GPT-4o-mini, O1-mini, and DeepSeek-R1 in social media engagement simulation regarding a major societal event.
  arXiv  Detail & Related papers  (2025-02-17T17:43:08Z)
• NewsInterview: a Dataset and a Playground to Evaluate LLMs' Ground Gap via Informational Interviews [65.35458530702442]
  We focus on journalistic interviews, a domain rich in grounding communication and abundant in data. We curate a dataset of 40,000 two-person informational interviews from NPR and CNN. LLMs are significantly less likely than human interviewers to use acknowledgements and to pivot to higher-level questions.
  arXiv  Detail & Related papers  (2024-11-21T01:37:38Z)
• Engagement-Driven Content Generation with Large Language Models [8.049552839071918]
  Large Language Models (LLMs) demonstrate significant persuasive capabilities in one-on-one interactions.<n>Their influence within social networks, where interconnected users and complex opinion dynamics pose unique challenges, remains underexplored.<n>This paper addresses the research question: emphCan LLMs generate meaningful content that maximizes user engagement on social networks?
  arXiv  Detail & Related papers  (2024-11-20T10:40:08Z)
• Robots in the Middle: Evaluating LLMs in Dispute Resolution [0.0]
  We investigate whether large language models (LLMs) are able to analyze dispute conversations, select suitable intervention types, and generate appropriate intervention messages. Our results demonstrate the potential of integrating AI in online dispute resolution (ODR) platforms.
  arXiv  Detail & Related papers  (2024-10-09T16:51:10Z)
• Automated test generation to evaluate tool-augmented LLMs as conversational AI agents [0.27309692684728615]
  We present a test generation pipeline to evaluate conversational AI agents. Our framework uses LLMs to generate diverse tests grounded on user-defined procedures. Our results show that while tool-augmented LLMs perform well in single interactions, they often struggle to handle complete conversations.
  arXiv  Detail & Related papers  (2024-09-24T09:57:43Z)
• Feedback Loops With Language Models Drive In-Context Reward Hacking [78.9830398771605]
  We show that feedback loops can cause in-context reward hacking (ICRH) We identify and study two processes that lead to ICRH: output-refinement and policy-refinement. As AI development accelerates, the effects of feedback loops will proliferate.
  arXiv  Detail & Related papers  (2024-02-09T18:59:29Z)
• LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models [56.25156596019168]
  This paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for large language models (LLMs) Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games.
  arXiv  Detail & Related papers  (2023-11-30T03:59:31Z)
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
  Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
  arXiv  Detail & Related papers  (2023-11-09T18:45:16Z)
• Harnessing the Power of LLMs: Evaluating Human-AI Text Co-Creation through the Lens of News Headline Generation [58.31430028519306]
  This study explores how humans can best leverage LLMs for writing and how interacting with these models affects feelings of ownership and trust in the writing process. While LLMs alone can generate satisfactory news headlines, on average, human control is needed to fix undesirable model outputs.
  arXiv  Detail & Related papers  (2023-10-16T15:11:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.