Can Large Language Models Detect Rumors on Social Media?
- URL: http://arxiv.org/abs/2402.03916v2
- Date: Thu, 8 Feb 2024 16:09:36 GMT
- Title: Can Large Language Models Detect Rumors on Social Media?
- Authors: Qiang Liu, Xiang Tao, Junfei Wu, Shu Wu, Liang Wang
- Abstract summary: We investigate to use Large Language Models (LLMs) for rumor detection on social media.
We propose an LLM-empowered Rumor Detection (LeRuD) approach, in which we design prompts to teach LLMs to reason over important clues in news and comments.
LeRuD outperforms several state-of-the-art rumor detection models by 3.2% to 7.7%.
- Score: 21.678652268122296
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this work, we investigate to use Large Language Models (LLMs) for rumor
detection on social media. However, it is challenging for LLMs to reason over
the entire propagation information on social media, which contains news
contents and numerous comments, due to LLMs may not concentrate on key clues in
the complex propagation information, and have trouble in reasoning when facing
massive and redundant information. Accordingly, we propose an LLM-empowered
Rumor Detection (LeRuD) approach, in which we design prompts to teach LLMs to
reason over important clues in news and comments, and divide the entire
propagation information into a Chain-of-Propagation for reducing LLMs' burden.
We conduct extensive experiments on the Twitter and Weibo datasets, and LeRuD
outperforms several state-of-the-art rumor detection models by 3.2% to 7.7%.
Meanwhile, by applying LLMs, LeRuD requires no data for training, and thus
shows more promising rumor detection ability in few-shot or zero-shot
scenarios.
Related papers
- Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data [10.922885479487066]
We study inductive out-of-context reasoning (OOCR) in which LLMs infer latent information from evidence distributed across training documents.
In one experiment we finetune an LLM on a corpus consisting only of distances between an unknown city and other known cities.
While OOCR succeeds in a range of cases, we also show that it is unreliable, particularly for smaller LLMs learning complex structures.
arXiv Detail & Related papers (2024-06-20T17:55:04Z) - Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs [60.40396361115776]
This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in large language models (LLMs) with a slim proxy model.
We employ a proxy model which has far fewer parameters, and take its answers as answers.
Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM.
arXiv Detail & Related papers (2024-02-19T11:11:08Z) - LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation [58.524237916836164]
We propose LEMMA: LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation.
Our method improves the accuracy over the top baseline LVLM by 7% and 13% on Twitter and Fakeddit datasets respectively.
arXiv Detail & Related papers (2024-02-19T08:32:27Z) - Large Language Models: A Survey [69.72787936480394]
Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks.
LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
arXiv Detail & Related papers (2024-02-09T05:37:09Z) - Disinformation Capabilities of Large Language Models [0.564232659769944]
This paper presents a study of the disinformation capabilities of the current generation of large language models (LLMs)
We evaluated the capabilities of 10 LLMs using 20 disinformation narratives.
We conclude that LLMs are able to generate convincing news articles that agree with dangerous disinformation narratives.
arXiv Detail & Related papers (2023-11-15T10:25:30Z) - Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong [35.64962031447787]
Large Language Models (LLMs) are increasingly used for accessing information on the web.
Our experiments with 80 crowdworkers compare language models with search engines (information retrieval systems) at facilitating fact-checking.
Users reading LLM explanations are significantly more efficient than those using search engines while achieving similar accuracy.
arXiv Detail & Related papers (2023-10-19T08:09:58Z) - Can LLM-Generated Misinformation Be Detected? [18.378744138365537]
Large Language Models (LLMs) can be exploited to generate misinformation.
A fundamental research question is: will LLM-generated misinformation cause more harm than human-written misinformation?
arXiv Detail & Related papers (2023-09-25T00:45:07Z) - Bad Actor, Good Advisor: Exploring the Role of Large Language Models in
Fake News Detection [22.658378054986624]
Large language models (LLMs) have shown remarkable performance in various tasks.
LLMs provide desirable multi-perspective rationales but still underperform the basic SLM, fine-tuned BERT.
We propose that current LLMs may not substitute fine-tuned SLMs in fake news detection but can be a good advisor for SLMs.
arXiv Detail & Related papers (2023-09-21T16:47:30Z) - Are Large Language Models Really Robust to Word-Level Perturbations? [68.60618778027694]
We propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools.
Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions.
Our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage.
arXiv Detail & Related papers (2023-09-20T09:23:46Z) - On the Risk of Misinformation Pollution with Large Language Models [127.1107824751703]
We investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation.
Our study reveals that LLMs can act as effective misinformation generators, leading to a significant degradation in the performance of Open-Domain Question Answering (ODQA) systems.
arXiv Detail & Related papers (2023-05-23T04:10:26Z) - Can Large Language Models Transform Computational Social Science? [79.62471267510963]
Large Language Models (LLMs) are capable of performing many language processing tasks zero-shot (without training data)
This work provides a road map for using LLMs as Computational Social Science tools.
arXiv Detail & Related papers (2023-04-12T17:33:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.