Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements

• URL: http://arxiv.org/abs/2402.10614v2
• Date: Fri, 7 Jun 2024 20:19:09 GMT
• Title: Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
• Authors: Ming Li, Jiuhai Chen, Lichang Chen, Tianyi Zhou,
• Abstract summary: We improve the controllability of LLMs in generating statements supporting an argument the user defined in the prompt. We develop a novel debate & tuning pipeline finetuning LLMs to generate the statements obtained via debate.
• Score: 30.970994382186944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Making LLMs speak for different, especially minority groups of people, and generate statements supporting their diverse or even controversial perspectives is critical to creating an inclusive environment. However, existing LLMs lack sufficient controllability to the stance of their generated content, which often contains inconsistent, neutral, or biased statements. In this paper, we improve the controllability of LLMs in generating statements supporting an argument the user defined in the prompt. We find that multi-round debates between two LLMs with opposite stances generate higher-quality and more salient statements for each, which are important training data to improve the controllability of LLMs. Motivated by this, we develop a novel debate & tuning (DEBATUNE) pipeline finetuning LLMs to generate the statements obtained via debate. To examine DEBATUNE, we curate the largest dataset of debate topics so far, which covers 710 controversial topics and corresponding arguments for each topic. Evaluations by the GPT-4 judge with a novel controversy controllability metric show that LLMs' capability of generating diverse perspectives is significantly improved by DEBATUNE. Moreover, such controllability can be generalized to unseen topics, generating high-quality statements supporting controversial arguments.

Related papers

• MArgE: Meshing Argumentative Evidence from Multiple Large Language Models for Justifiable Claim Verification [12.449402503089164]
  We introduce MArgE, a novel framework to provide formal structure to the evidence from each large language model.<n>We show experimentally that MArgE can significantly outperform single LLMs.
  arXiv  Detail & Related papers  (2025-08-04T16:40:02Z)
• Multiple LLM Agents Debate for Equitable Cultural Alignment [52.01956042197423]
  We introduce a Multi-Agent Debate framework, where two LLM-based agents debate over a cultural scenario and collaboratively reach a final decision.<n>We evaluate these approaches on 7 open-weight LLMs (and 21 LLM combinations) using the NormAd-ETI benchmark for social etiquette norms in 75 countries.<n>Experiments show that debate improves both overall accuracy and cultural group parity over single-LLM baselines.
  arXiv  Detail & Related papers  (2025-05-30T15:01:52Z)
• Arbiters of Ambivalence: Challenges of Using LLMs in No-Consensus Tasks [52.098988739649705]
  This study examines the biases and limitations of LLMs in three roles: answer generator, judge, and debater.<n>We develop a no-consensus'' benchmark by curating examples that encompass a variety of a priori ambivalent scenarios.<n>Our results show that while LLMs can provide nuanced assessments when generating open-ended answers, they tend to take a stance on no-consensus topics when employed as judges or debaters.
  arXiv  Detail & Related papers  (2025-05-28T01:31:54Z)
• Don't Stop the Multi-Party! On Generating Synthetic Multi-Party Conversations with Constraints [11.566214724241798]
  Multi-Party Conversations (MPCs) are widely studied across disciplines, with social media as a primary data source due to their accessibility. This work explores the feasibility of generating diverse MPCs with instruction-tuned Large Language Models.
  arXiv  Detail & Related papers  (2025-02-19T10:10:43Z)
• Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation [65.92001420372007]
  This paper systematically evaluates state-of-the-art MLLMs across diverse benchmarks. We introduce the first benchmark GaslightingBench, specifically designed to evaluate the vulnerability of MLLMs to negation arguments.
  arXiv  Detail & Related papers  (2025-01-31T10:37:48Z)
• DebateQA: Evaluating Question Answering on Debatable Knowledge [13.199937786970027]
  We introduce DebateQA, a dataset of 2,941 debatable questions. We develop two metrics: Perspective Diversity and Dispute Awareness. Using DebateQA with two metrics, we assess 12 popular large language models.
  arXiv  Detail & Related papers  (2024-08-02T17:54:34Z)
• DebUnc: Mitigating Hallucinations in Large Language Model Agent Communication with Uncertainty Estimations [52.242449026151846]
  DebUnc is a multi-agent debate framework that uses uncertainty metrics to assess agent confidence levels. We adapted the attention mechanism to adjust token weights based on confidence levels. Our evaluations show that attention-based methods are particularly effective.
  arXiv  Detail & Related papers  (2024-07-08T22:15:01Z)
• Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs [45.38821594541265]
  Large Language Models (LLMs) excel in various natural language processing tasks but struggle with hallucination issues. We propose a CounterFactual Multi-Agent Debate (CFMAD) framework to override LLMs' inherent biases for answer inspection.
  arXiv  Detail & Related papers  (2024-06-17T13:21:23Z)
• ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs [95.15814662348245]
  Compositional Reasoning (CR) entails grasping the significance of attributes, relations, and word order. Recent Vision-Language Models (VLMs) have demonstrated remarkable proficiency in such reasoning tasks.
  arXiv  Detail & Related papers  (2024-06-12T12:54:27Z)
• Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like One [43.37522760105383]
  We develop FairThinking, a pipeline designed to automatically generate roles that enable LLMs to articulate diverse perspectives for fair expressions. To evaluate FairThinking, we create a dataset with a thousand items covering three fairness-related topics and conduct experiments on GPT-3.5, GPT-4, Llama2, and Mistral.
  arXiv  Detail & Related papers  (2024-02-19T14:02:22Z)
• Prompt Highlighter: Interactive Control for Multi-Modal LLMs [50.830448437285355]
  This study targets a critical aspect of multi-modal LLMs' (LLMs&VLMs) inference: explicit controllable text generation. We introduce a novel inference method, Prompt Highlighter, which enables users to highlight specific prompt spans to interactively control the focus during generation. We find that, during inference, guiding the models with highlighted tokens through the attention weights leads to more desired outputs.
  arXiv  Detail & Related papers  (2023-12-07T13:53:29Z)
• Let Models Speak Ciphers: Multiagent Debate through Embeddings [84.20336971784495]
  We introduce CIPHER (Communicative Inter-Model Protocol Through Embedding Representation) to address this issue. By deviating from natural language, CIPHER offers an advantage of encoding a broader spectrum of information without any modification to the model weights. This showcases the superiority and robustness of embeddings as an alternative "language" for communication among LLMs.
  arXiv  Detail & Related papers  (2023-10-10T03:06:38Z)
• Are Large Language Models Really Robust to Word-Level Perturbations? [68.60618778027694]
  We propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools. Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions. Our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage.
  arXiv  Detail & Related papers  (2023-09-20T09:23:46Z)
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate [85.3444184685235]
  We propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution. Our framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation.
  arXiv  Detail & Related papers  (2023-05-30T15:25:45Z)
• Prompting Large Language Models for Counterfactual Generation: An Empirical Study [13.506528217009507]
  Large language models (LLMs) have made remarkable progress in a wide range of natural language understanding and generation tasks. We present a comprehensive evaluation framework on various types of NLU tasks, which covers all key factors in determining LLMs' capability of generating counterfactuals.
  arXiv  Detail & Related papers  (2023-05-24T06:44:32Z)
• Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate [19.887103433032774]
  Large language models (LLMs) have shown impressive performance in complex reasoning tasks. This work explores testing LLMs' reasoning by engaging with them in a debate-like conversation. We find that despite their impressive performance, LLMs like ChatGPT cannot maintain their beliefs in truth for a significant portion of examples.
  arXiv  Detail & Related papers  (2023-05-22T15:47:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.