Disinformation Detection: An Evolving Challenge in the Age of LLMs

• URL: http://arxiv.org/abs/2309.15847v1
• Date: Mon, 25 Sep 2023 22:12:50 GMT
• Title: Disinformation Detection: An Evolving Challenge in the Age of LLMs
• Authors: Bohan Jiang, Zhen Tan, Ayushi Nirmal, Huan Liu
• Abstract summary: Large Language Models (LLMs) can generate highly persuasive yet misleading content. LLMs can be exploited to serve as a robust defense against advanced disinformation. A holistic exploration for the formation and detection of disinformation is conducted to foster this line of research.
• Score: 16.46484369516341
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advent of generative Large Language Models (LLMs) such as ChatGPT has catalyzed transformative advancements across multiple domains. However, alongside these advancements, they have also introduced potential threats. One critical concern is the misuse of LLMs by disinformation spreaders, leveraging these models to generate highly persuasive yet misleading content that challenges the disinformation detection system. This work aims to address this issue by answering three research questions: (1) To what extent can the current disinformation detection technique reliably detect LLM-generated disinformation? (2) If traditional techniques prove less effective, can LLMs themself be exploited to serve as a robust defense against advanced disinformation? and, (3) Should both these strategies falter, what novel approaches can be proposed to counter this burgeoning threat effectively? A holistic exploration for the formation and detection of disinformation is conducted to foster this line of research.

Related papers

• Catching Chameleons: Detecting Evolving Disinformation Generated using Large Language Models [16.408611714514976]
  We propose DELD (Detecting Evolving LLM-generated Disinformation), a parameter-efficient approach that jointly leverages the general fact-checking capabilities of pre-trained language models. Our experiments show that textitDELD significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-26T00:21:39Z)
• The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
  Large Language Models (LLMs) are foundational to AI advancements. LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information. Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
  arXiv  Detail & Related papers  (2024-03-23T09:26:15Z)
• LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation [58.524237916836164]
  We propose LEMMA: LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation. Our method improves the accuracy over the top baseline LVLM by 7% and 13% on Twitter and Fakeddit datasets respectively.
  arXiv  Detail & Related papers  (2024-02-19T08:32:27Z)
• Disinformation Capabilities of Large Language Models [0.564232659769944]
  This paper presents a study of the disinformation capabilities of the current generation of large language models (LLMs) We evaluated the capabilities of 10 LLMs using 20 disinformation narratives. We conclude that LLMs are able to generate convincing news articles that agree with dangerous disinformation narratives.
  arXiv  Detail & Related papers  (2023-11-15T10:25:30Z)
• Insights into Classifying and Mitigating LLMs' Hallucinations [48.04565928175536]
  This paper delves into the underlying causes of AI hallucination and elucidates its significance in artificial intelligence. We explore potential strategies to mitigate hallucinations, aiming to enhance the overall reliability of large language models.
  arXiv  Detail & Related papers  (2023-11-14T12:30:28Z)
• A Survey on Detection of LLMs-Generated Content [97.87912800179531]
  The ability to detect LLMs-generated content has become of paramount importance. We aim to provide a detailed overview of existing detection strategies and benchmarks. We also posit the necessity for a multi-faceted approach to defend against various attacks.
  arXiv  Detail & Related papers  (2023-10-24T09:10:26Z)
• A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions [39.36381851190369]
  There is an imperative need to develop detectors that can detect LLM-generated text. This is crucial to mitigate potential misuse of LLMs and safeguard realms like artistic expression and social networks from harmful influence of LLM-generated content. The detector techniques have witnessed notable advancements recently, propelled by innovations in watermarking techniques, statistics-based detectors, neural-base detectors, and human-assisted methods.
  arXiv  Detail & Related papers  (2023-10-23T09:01:13Z)
• Can LLM-Generated Misinformation Be Detected? [18.378744138365537]
  Large Language Models (LLMs) can be exploited to generate misinformation. A fundamental research question is: will LLM-generated misinformation cause more harm than human-written misinformation?
  arXiv  Detail & Related papers  (2023-09-25T00:45:07Z)
• Red Teaming Language Model Detectors with Language Models [114.36392560711022]
  Large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. Recent works have proposed algorithms to detect LLM-generated text and protect LLMs. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation.
  arXiv  Detail & Related papers  (2023-05-31T10:08:37Z)
• On the Risk of Misinformation Pollution with Large Language Models [127.1107824751703]
  We investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation. Our study reveals that LLMs can act as effective misinformation generators, leading to a significant degradation in the performance of Open-Domain Question Answering (ODQA) systems.
  arXiv  Detail & Related papers  (2023-05-23T04:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.