Emotional Manipulation Through Prompt Engineering Amplifies Disinformation Generation in AI Large Language Models

• URL: http://arxiv.org/abs/2403.03550v1
• Date: Wed, 6 Mar 2024 08:50:25 GMT
• Title: Emotional Manipulation Through Prompt Engineering Amplifies Disinformation Generation in AI Large Language Models
• Authors: Rasita Vinay, Giovanni Spitale, Nikola Biller-Andorno, Federico Germani
• Abstract summary: This study investigates the generation of synthetic disinformation by OpenAI's Large Language Models (LLMs) through prompt engineering and explores their responsiveness to emotional prompting. Our findings, based on a corpus of 19, synthetic disinformation social media posts, reveal that all LLMs by OpenAI can successfully produce disinformation. When prompted politely, all examined LLMs consistently generate disinformation at a high frequency. However, when prompted impolitely, the frequency of disinformation production diminishes, as the models often refuse to generate disinformation and instead caution users that the tool is not intended for such purposes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This study investigates the generation of synthetic disinformation by OpenAI's Large Language Models (LLMs) through prompt engineering and explores their responsiveness to emotional prompting. Leveraging various LLM iterations using davinci-002, davinci-003, gpt-3.5-turbo and gpt-4, we designed experiments to assess their success in producing disinformation. Our findings, based on a corpus of 19,800 synthetic disinformation social media posts, reveal that all LLMs by OpenAI can successfully produce disinformation, and that they effectively respond to emotional prompting, indicating their nuanced understanding of emotional cues in text generation. When prompted politely, all examined LLMs consistently generate disinformation at a high frequency. Conversely, when prompted impolitely, the frequency of disinformation production diminishes, as the models often refuse to generate disinformation and instead caution users that the tool is not intended for such purposes. This research contributes to the ongoing discourse surrounding responsible development and application of AI technologies, particularly in mitigating the spread of disinformation and promoting transparency in AI-generated content.

Related papers

• Information Retrieval in the Age of Generative AI: The RGB Model [77.96475639967431]
  This paper presents a novel quantitative approach to shed light on the complex information dynamics arising from the growing use of generative AI tools. We propose a model to characterize the generation, indexing, and dissemination of information in response to new topics. Our findings suggest that the rapid pace of generative AI adoption, combined with increasing user reliance, can outpace human verification, escalating the risk of inaccurate information proliferation.
  arXiv  Detail & Related papers  (2025-04-29T10:21:40Z)
• DoYouTrustAI: A Tool to Teach Students About AI Misinformation and Prompt Engineering [2.3020018305241337]
  DoYouTrustAI is a web-based application that helps students enhance critical thinking by identifying misleading information in LLM responses about major historical figures. The tool lets users select familiar individuals for testing to reduce random guessing and presents misinformation alongside known facts to maintain believability. It also provides pre-configured prompt instructions to show how different prompts affect AI responses.
  arXiv  Detail & Related papers  (2025-03-22T19:11:57Z)
• Prompt Sentiment: The Catalyst for LLM Change [0.29998889086656577]
  This study systematically examines how sentiment variations in prompts affect large language models (LLMs) Our analysis spans six AI-driven applications, including content generation, conversational AI, legal and financial analysis, healthcare AI, creative writing, and technical documentation. Our findings reveal that prompt sentiment significantly influences model responses, with negative prompts often reducing factual accuracy and amplifying bias, while positive prompts tend to increase verbosity and sentiment propagation.
  arXiv  Detail & Related papers  (2025-03-14T06:25:21Z)
• Are Large Language Models Ready for Business Integration? A Study on Generative AI Adoption [0.6144680854063939]
  This research examines the readiness of other Large Language Models (LLMs) such as Google Gemini for potential business applications. A dataset of 42,654 reviews from distinct Disneyland branches was employed. Results presented a spectrum of responses, including 75% successful simplifications, 25% errors, and instances of model self-reference.
  arXiv  Detail & Related papers  (2025-01-28T21:01:22Z)
• Deploying Large Language Models With Retrieval Augmented Generation [0.21485350418225244]
  Retrieval Augmented Generation has emerged as a key approach for integrating knowledge from data sources outside of the large language model's training set. We present insights from the development and field-testing of a pilot project that integrates LLMs with RAG for information retrieval.
  arXiv  Detail & Related papers  (2024-11-07T22:11:51Z)
• O1 Replication Journey: A Strategic Progress Report -- Part 1 [52.062216849476776]
  This paper introduces a pioneering approach to artificial intelligence research, embodied in our O1 Replication Journey. Our methodology addresses critical challenges in modern AI research, including the insularity of prolonged team-based projects. We propose the journey learning paradigm, which encourages models to learn not just shortcuts, but the complete exploration process.
  arXiv  Detail & Related papers  (2024-10-08T15:13:01Z)
• The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
  Large Language Models (LLMs) are foundational to AI advancements. LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information. Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
  arXiv  Detail & Related papers  (2024-03-23T09:26:15Z)
• A Closer Look at the Limitations of Instruction Tuning [52.587607091917214]
  We show that Instruction Tuning (IT) fails to enhance knowledge or skills in large language models (LLMs) We also show that popular methods to improve IT do not lead to performance improvements over a simple LoRA fine-tuned model. Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets.
  arXiv  Detail & Related papers  (2024-02-03T04:45:25Z)
• Generative AI in Writing Research Papers: A New Type of Algorithmic Bias and Uncertainty in Scholarly Work [0.38850145898707145]
  Large language models (LLMs) and generative AI tools present challenges in identifying and addressing biases. generative AI tools are susceptible to goal misgeneralization, hallucinations, and adversarial attacks such as red teaming prompts. We find that incorporating generative AI in the process of writing research manuscripts introduces a new type of context-induced algorithmic bias.
  arXiv  Detail & Related papers  (2023-12-04T04:05:04Z)
• RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledge [69.79676144482792]
  This study aims to evaluate the ability of LLMs to distinguish reliable information from external knowledge. Our benchmark consists of two tasks, Question Answering and Text Generation, and for each task, we provide models with a context containing counterfactual information.
  arXiv  Detail & Related papers  (2023-11-14T13:24:19Z)
• Insights into Classifying and Mitigating LLMs' Hallucinations [48.04565928175536]
  This paper delves into the underlying causes of AI hallucination and elucidates its significance in artificial intelligence. We explore potential strategies to mitigate hallucinations, aiming to enhance the overall reliability of large language models.
  arXiv  Detail & Related papers  (2023-11-14T12:30:28Z)
• The Age of Synthetic Realities: Challenges and Opportunities [85.058932103181]
  We highlight the crucial need for the development of forensic techniques capable of identifying harmful synthetic creations and distinguishing them from reality. Our focus extends to various forms of media, such as images, videos, audio, and text, as we examine how synthetic realities are crafted and explore approaches to detecting these malicious creations. This study is of paramount importance due to the rapid progress of AI generative techniques and their impact on the fundamental principles of Forensic Science.
  arXiv  Detail & Related papers  (2023-06-09T15:55:10Z)
• Detection of Fake Generated Scientific Abstracts [0.9525711971667679]
  The academic community has expressed concerns regarding the difficulty of discriminating between what is real and what is artificially generated. In this study, we utilize the GPT-3 model to generate scientific paper abstracts through Artificial Intelligence. We explore various text representation methods when combined with Machine Learning models with the aim of identifying machine-written text.
  arXiv  Detail & Related papers  (2023-04-12T20:20:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.