Large Language Models as Misleading Assistants in Conversation

• URL: http://arxiv.org/abs/2407.11789v1
• Date: Tue, 16 Jul 2024 14:45:22 GMT
• Title: Large Language Models as Misleading Assistants in Conversation
• Authors: Betty Li Hou, Kejian Shi, Jason Phang, James Aung, Steven Adler, Rosie Campbell,
• Abstract summary: We investigate the ability of Large Language Models (LLMs) to be deceptive in the context of providing assistance on a reading comprehension task. We compare outcomes of (1) when the model is prompted to provide truthful assistance, (2) when it is prompted to be subtly misleading, and (3) when it is prompted to argue for an incorrect answer.
• Score: 8.557086720583802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are able to provide assistance on a wide range of information-seeking tasks. However, model outputs may be misleading, whether unintentionally or in cases of intentional deception. We investigate the ability of LLMs to be deceptive in the context of providing assistance on a reading comprehension task, using LLMs as proxies for human users. We compare outcomes of (1) when the model is prompted to provide truthful assistance, (2) when it is prompted to be subtly misleading, and (3) when it is prompted to argue for an incorrect answer. Our experiments show that GPT-4 can effectively mislead both GPT-3.5-Turbo and GPT-4, with deceptive assistants resulting in up to a 23% drop in accuracy on the task compared to when a truthful assistant is used. We also find that providing the user model with additional context from the passage partially mitigates the influence of the deceptive model. This work highlights the ability of LLMs to produce misleading information and the effects this may have in real-world situations.

Related papers

• Delving into the Reversal Curse: How Far Can Large Language Models Generalize? [40.64539467276017]
  A prime example is the recently debated "reversal curse", which surfaces when models, having been trained on the fact "A is B", struggle to generalize this knowledge to infer that "B is A" In this paper, we examine the manifestation of the reversal curse across various tasks and delve into both the generalization abilities and the problem-solving mechanisms of LLMs.
  arXiv  Detail & Related papers  (2024-10-24T14:55:09Z)
• LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users [17.739596091065856]
  We investigate how the quality of Large Language Models responses changes in terms of information accuracy, truthfulness, and refusals depending on user traits. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US.
  arXiv  Detail & Related papers  (2024-06-25T17:24:07Z)
• LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements [59.71218039095155]
  Task of reading comprehension (RC) provides a primary means to assess language models' natural language understanding (NLU) capabilities. If the context aligns with the models' internal knowledge, it is hard to discern whether the models' answers stem from context comprehension or from internal information. To address this issue, we suggest to use RC on imaginary data, based on fictitious facts and entities.
  arXiv  Detail & Related papers  (2024-04-09T13:08:56Z)
• The Earth is Flat? Unveiling Factual Errors in Large Language Models [89.94270049334479]
  Large Language Models (LLMs) like ChatGPT are in various applications due to their extensive knowledge from pre-training and fine-tuning. Despite this, they are prone to generating factual and commonsense errors, raising concerns in critical areas like healthcare, journalism, and education. We introduce a novel, automatic testing framework, FactChecker, aimed at uncovering factual inaccuracies in LLMs.
  arXiv  Detail & Related papers  (2024-01-01T14:02:27Z)
• The Perils & Promises of Fact-checking with Large Language Models [55.869584426820715]
  Large Language Models (LLMs) are increasingly trusted to write academic papers, lawsuits, and news articles. We evaluate the use of LLM agents in fact-checking by having them phrase queries, retrieve contextual data, and make decisions. Our results show the enhanced prowess of LLMs when equipped with contextual information. While LLMs show promise in fact-checking, caution is essential due to inconsistent accuracy.
  arXiv  Detail & Related papers  (2023-10-20T14:49:47Z)
• Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong [35.64962031447787]
  Large Language Models (LLMs) are increasingly used for accessing information on the web. Our experiments with 80 crowdworkers compare language models with search engines (information retrieval systems) at facilitating fact-checking. Users reading LLM explanations are significantly more efficient than those using search engines while achieving similar accuracy.
  arXiv  Detail & Related papers  (2023-10-19T08:09:58Z)
• Evaluating Language Models for Mathematics through Interactions [116.67206980096513]
  We introduce CheckMate, a prototype platform for humans to interact with and evaluate large language models (LLMs) We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics. We derive a taxonomy of human behaviours and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness.
  arXiv  Detail & Related papers  (2023-06-02T17:12:25Z)
• Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis [103.89753784762445]
  Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT) This paper systematically investigates the advantages and challenges of LLMs for MMT. We thoroughly evaluate eight popular LLMs, including ChatGPT and GPT-4.
  arXiv  Detail & Related papers  (2023-04-10T15:51:30Z)
• Prompting GPT-3 To Be Reliable [117.23966502293796]
  This work decomposes reliability into four facets: generalizability, fairness, calibration, and factuality. We find that GPT-3 outperforms smaller-scale supervised models by large margins on all these facets.
  arXiv  Detail & Related papers  (2022-10-17T14:52:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.