Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?

• URL: http://arxiv.org/abs/2311.09702v3
• Date: Fri, 5 Apr 2024 18:08:51 GMT
• Title: Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?
• Authors: Bangzheng Li, Ben Zhou, Fei Wang, Xingyu Fu, Dan Roth, Muhao Chen,
• Abstract summary: This work studies a specific type of hallucination induced by semantic associations. To quantify this phenomenon, we propose a novel probing method and benchmark called EureQA.
• Score: 73.454943870226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the recent advancement in large language models (LLMs) and their high performances across numerous benchmarks, recent research has unveiled that LLMs suffer from hallucinations and unfaithful reasoning. This work studies a specific type of hallucination induced by semantic associations. Specifically, we investigate to what extent LLMs take shortcuts from certain keyword/entity biases in the prompt instead of following the correct reasoning path. To quantify this phenomenon, we propose a novel probing method and benchmark called EureQA. We start from questions that LLMs will answer correctly with utmost certainty, and mask the important entity with evidence sentence recursively, asking models to find masked entities according to a chain of evidence before answering the question. During the construction of the evidence, we purposefully replace semantic clues (entities) that may lead to the correct answer with distractor clues (evidence) that will not directly lead to the correct answer but require a chain-like reasoning process. We evaluate if models can follow the correct reasoning chain instead of short-cutting through distractor clues. We find that existing LLMs lack the necessary capabilities to follow correct reasoning paths and resist the attempt of greedy shortcuts. We show that the distractor semantic associations often lead to model hallucination, which is strong evidence that questions the validity of current LLM reasoning.

Related papers

• DecoPrompt : Decoding Prompts Reduces Hallucinations when Large Language Models Meet False Premises [28.72485319617863]
  We propose a new prompting algorithm, named DecoPrompt, to mitigate hallucination. DecoPrompt leverages LLMs to "decode" the false-premise prompts without really eliciting hallucination output from LLMs. We perform experiments on two datasets, demonstrating that DecoPrompt can reduce hallucinations effectively on outputs from different LLMs.
  arXiv  Detail & Related papers  (2024-11-12T00:48:01Z)
• Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
  We investigate the causal reasoning abilities of large language models (LLMs) through the representative problem of inferring causal relationships from narratives. We find that even state-of-the-art language models rely on unreliable shortcuts, both in terms of the narrative presentation and their parametric knowledge.
  arXiv  Detail & Related papers  (2024-10-31T12:48:58Z)
• Automatic Curriculum Expert Iteration for Reliable LLM Reasoning [60.60318625779015]
  Hallucinations (i.e., generating plausible but inaccurate content) and laziness (i.e. excessive refusals or defaulting to "I don't know") persist as major challenges in LLM reasoning. Current efforts to reduce hallucinations primarily focus on factual errors in knowledge-grounded tasks, often neglecting hallucinations related to faulty reasoning. We propose Automatic Curriculum Expert Iteration (Auto-CEI) to enhance LLM reasoning and align responses to the model's capabilities.
  arXiv  Detail & Related papers  (2024-10-10T05:43:07Z)
• Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? [6.525065859315515]
  We investigate whether Large Language Models (LLMs) are prone to exploiting simplifying cues in multi-hop reasoning benchmarks. Motivated by this finding, we propose a challenging multi-hop reasoning benchmark, by generating seemingly plausible multi-hop reasoning chains. We find that their performance to perform multi-hop reasoning is affected, as indicated by up to 45% relative decrease in F1 score when presented with such seemingly plausible alternatives.
  arXiv  Detail & Related papers  (2024-09-08T19:22:58Z)
• Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models [0.0]
  Large language models (LLMs) have generated significant attention since their inception, finding applications across various academic and industrial domains. LLMs often suffer from the "hallucination problem", where outputs, though grammatically and logically coherent, lack factual accuracy or are entirely fabricated.
  arXiv  Detail & Related papers  (2024-08-09T14:34:32Z)
• Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models [52.957842999317506]
  Object hallucination refers to the phenomenon that the LVLMs claim non-existent objects in the image. We propose a Logical Closed Loop-based framework for Object Hallucination Detection and Mitigation, namely LogicCheckGPT. As a plug-and-play method, it can be seamlessly applied to all existing LVLMs.
  arXiv  Detail & Related papers  (2024-02-18T15:28:39Z)
• The ART of LLM Refinement: Ask, Refine, and Trust [85.75059530612882]
  We propose a reasoning with refinement objective called ART: Ask, Refine, and Trust. It asks necessary questions to decide when an LLM should refine its output. It achieves a performance gain of +5 points over self-refinement baselines.
  arXiv  Detail & Related papers  (2023-11-14T07:26:32Z)
• A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning [73.77088902676306]
  We take a closer look at the self-verification abilities of large language models (LLMs) in the context of logical reasoning. Our main findings suggest that existing LLMs could struggle to identify fallacious reasoning steps accurately and may fall short of guaranteeing the validity of self-verification methods.
  arXiv  Detail & Related papers  (2023-11-14T07:13:10Z)
• Boosting Language Models Reasoning with Chain-of-Knowledge Prompting [18.326858925174605]
  Chain-of-Knowledge (CoK) prompting aims at eliciting explicit pieces of knowledge evidence in the form of structure triple. Benefiting from CoK, we additionally introduce a F2-Verification method to estimate the reliability of the reasoning chains. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.
  arXiv  Detail & Related papers  (2023-06-10T12:42:36Z)
• Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate [19.887103433032774]
  Large language models (LLMs) have shown impressive performance in complex reasoning tasks. This work explores testing LLMs' reasoning by engaging with them in a debate-like conversation. We find that despite their impressive performance, LLMs like ChatGPT cannot maintain their beliefs in truth for a significant portion of examples.
  arXiv  Detail & Related papers  (2023-05-22T15:47:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.