Breaking Event Rumor Detection via Stance-Separated Multi-Agent Debate

• URL: http://arxiv.org/abs/2412.04859v1
• Date: Fri, 06 Dec 2024 08:52:30 GMT
• Title: Breaking Event Rumor Detection via Stance-Separated Multi-Agent Debate
• Authors: Mingqing Zhang, Haisong Gong, Qiang Liu, Shu Wu, Liang Wang,
• Abstract summary: Leveraging large language models (LLMs) for rumor detection holds significant promise. We propose the Stance Separated Multi-Agent Debate (S2MAD) to address this issue. Our proposed model outperforms state-of-the-art methods in terms of performance.
• Score: 21.342632695285364
• License:
• Abstract: The rapid spread of rumors on social media platforms during breaking events severely hinders the dissemination of the truth. Previous studies reveal that the lack of annotated resources hinders the direct detection of unforeseen breaking events not covered in yesterday's news. Leveraging large language models (LLMs) for rumor detection holds significant promise. However, it is challenging for LLMs to provide comprehensive responses to complex or controversial issues due to limited diversity. In this work, we propose the Stance Separated Multi-Agent Debate (S2MAD) to address this issue. Specifically, we firstly introduce Stance Separation, categorizing comments as either supporting or opposing the original claim. Subsequently, claims are classified as subjective or objective, enabling agents to generate reasonable initial viewpoints with different prompt strategies for each type of claim. Debaters then follow specific instructions through multiple rounds of debate to reach a consensus. If a consensus is not reached, a judge agent evaluates the opinions and delivers a final verdict on the claim's veracity. Extensive experiments conducted on two real-world datasets demonstrate that our proposed model outperforms state-of-the-art methods in terms of performance and effectively improves the performance of LLMs in breaking event rumor detection.

Related papers

• Agentic Verification for Ambiguous Query Disambiguation [42.238086712267396]
  We tackle the challenge of disambiguating queries in retrieval-augmented generation (RAG) to diverse yet answerable interpretations. We propose a joint approach to unify diversification with verification by incorporating feedback from retriever and generator early on. We validate the efficiency and effectiveness of our method on the widely adopted ASQA benchmark to achieve diverse yet verifiable interpretations.
  arXiv  Detail & Related papers  (2025-02-14T18:31:39Z)
• Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges [52.96987928118327]
  We find that embedding models for retrieval, rerankers, and large language model (LLM) relevance judges are vulnerable to content injection attacks. We identify two primary threats: (1) inserting unrelated or harmful content within passages that still appear deceptively "relevant", and (2) inserting entire queries or key query terms into passages to boost their perceived relevance. Our study systematically examines the factors that influence an attack's success, such as the placement of injected content and the balance between relevant and non-relevant material.
  arXiv  Detail & Related papers  (2025-01-30T18:02:15Z)
• DebUnc: Mitigating Hallucinations in Large Language Model Agent Communication with Uncertainty Estimations [52.242449026151846]
  DebUnc is a multi-agent debate framework that uses uncertainty metrics to assess agent confidence levels. We adapted the attention mechanism to adjust token weights based on confidence levels. Our evaluations show that attention-based methods are particularly effective.
  arXiv  Detail & Related papers  (2024-07-08T22:15:01Z)
• Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs [45.38821594541265]
  Large Language Models (LLMs) excel in various natural language processing tasks but struggle with hallucination issues. We propose a CounterFactual Multi-Agent Debate (CFMAD) framework to override LLMs' inherent biases for answer inspection.
  arXiv  Detail & Related papers  (2024-06-17T13:21:23Z)
• Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework [41.47029501736853]
  We propose a Markov Chain-based multi-agent debate verification framework to enhance hallucination detection accuracy in concise claims. Our method integrates the fact-checking process, including claim detection, evidence retrieval, and multi-agent verification.
  arXiv  Detail & Related papers  (2024-06-05T08:59:45Z)
• Preemptive Answer "Attacks" on Chain-of-Thought Reasoning [7.233752893356647]
  Large language models (LLMs) showcase impressive reasoning capabilities when coupled with Chain-of-Thought prompting. In this paper, we introduce a novel scenario termed preemptive answers, where the LLM obtains an answer before engaging in reasoning. Experiments reveal that preemptive answers significantly impair the model's reasoning capability across various CoT methods and a broad spectrum of datasets.
  arXiv  Detail & Related papers  (2024-05-31T15:15:04Z)
• A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning [53.35861580821777]
  The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images. To address the issue, we propose a deductive (top-down) debating approach called Blueprint Debate on Graphs (BDoG) In BDoG, debates are confined to a blueprint graph to prevent opinion trivialization through world-level summarization. Moreover, by storing evidence in branches within the graph, BDoG mitigates distractions caused by frequent but irrelevant concepts.
  arXiv  Detail & Related papers  (2024-03-22T06:03:07Z)
• Sentiment Analysis through LLM Negotiations [58.67939611291001]
  A standard paradigm for sentiment analysis is to rely on a singular LLM and makes the decision in a single round. This paper introduces a multi-LLM negotiation framework for sentiment analysis.
  arXiv  Detail & Related papers  (2023-11-03T12:35:29Z)
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate [85.3444184685235]
  We propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution. Our framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation.
  arXiv  Detail & Related papers  (2023-05-30T15:25:45Z)
• Characterizing the adversarial vulnerability of speech self-supervised learning [95.03389072594243]
  We make the first attempt to investigate the adversarial vulnerability of such paradigm under the attacks from both zero-knowledge adversaries and limited-knowledge adversaries. The experimental results illustrate that the paradigm proposed by SUPERB is seriously vulnerable to limited-knowledge adversaries.
  arXiv  Detail & Related papers  (2021-11-08T08:44:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.