SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully

• URL: http://arxiv.org/abs/2401.05930v4
• Date: Mon, 07 Oct 2024 09:58:48 GMT
• Title: SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully
• Authors: Jushi Kai, Tianhang Zhang, Hai Hu, Zhouhan Lin,
• Abstract summary: We propose an inference-time method, Self-Highlighted Hesitation (SH2), to help large language models decode more truthfully. Experimental results demonstrate that our SH2 can effectively help LLMs elicit factual knowledge and distinguish hallucinated contexts.
• Score: 9.796854466591942
• License:
• Abstract: Large language models (LLMs) demonstrate great performance in text generation. However, LLMs are still suffering from hallucinations. In this work, we propose an inference-time method, Self-Highlighted Hesitation (SH2), to help LLMs decode more truthfully. SH2 is based on a simple fact rooted in information theory that for an LLM, the tokens predicted with lower probabilities are prone to be more informative than others. Our analysis shows that the tokens assigned with lower probabilities by an LLM are more likely to be closely related to factual information, such as nouns, proper nouns, and adjectives. Therefore, we propose to ''highlight'' the factual information by selecting the tokens with the lowest probabilities and concatenating them to the original context, thus forcing the model to repeatedly read and hesitate on these tokens before generation. During decoding, we also adopt contrastive decoding to emphasize the difference in the output probabilities brought by the hesitation. Experimental results demonstrate that our SH2, requiring no additional data or models, can effectively help LLMs elicit factual knowledge and distinguish hallucinated contexts. Significant and consistent improvements are achieved by SH2 for LLaMA-7b, LLaMA2-7b and Mistral-7b on multiple hallucination tasks.

Related papers

• Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only [37.36302216137465]
  We use methods based on self-consistency, token probabilities, and LLM-as-a-judge to elicit wrong-over-wrong preferences. Experiments show that LLMs do have preliminary capability in distinguishing various shades of wrong, achieving up to 20.9% higher performance than random guess.
  arXiv  Detail & Related papers  (2024-10-14T20:01:52Z)
• Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach [0.0]
  Large Language Models (LLMs) produce inaccurate outputs, also known as hallucinations. This paper introduces a supervised learning approach employing only four numerical features derived from tokens and vocabulary probabilities obtained from other evaluators. The method yields promising results, surpassing state-of-the-art outcomes in multiple tasks across three different benchmarks.
  arXiv  Detail & Related papers  (2024-05-30T03:00:47Z)
• FLAME: Factuality-Aware Alignment for Large Language Models [86.76336610282401]
  The conventional alignment process fails to enhance the factual accuracy of large language models (LLMs) We identify factors that lead to hallucination in both alignment steps: supervised fine-tuning (SFT) and reinforcement learning (RL) We propose factuality-aware alignment, comprised of factuality-aware SFT and factuality-aware RL through direct preference optimization.
  arXiv  Detail & Related papers  (2024-05-02T17:54:54Z)
• "Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing [10.20632187568563]
  Hallucination has emerged as the most vulnerable aspect of contemporary Large Language Models (LLMs) In this paper, we introduce the Sorry, Come Again (SCA) prompting, aimed to avoid LLM hallucinations. We provide an in-depth analysis of linguistic nuances: formality, readability, and concreteness of prompts for 21 LLMs. We propose an optimal paraphrasing technique to identify the most comprehensible paraphrase of a given prompt.
  arXiv  Detail & Related papers  (2024-03-27T19:45:09Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations [52.43593893122206]
  Alignedcot is an in-context learning technique for invoking Large Language Models. It achieves consistent and correct step-wise prompts in zero-shot scenarios. We conduct experiments on mathematical reasoning and commonsense reasoning.
  arXiv  Detail & Related papers  (2023-11-22T17:24:21Z)
• Are Large Language Models Temporally Grounded? [38.481606493496514]
  We provide Large language models (LLMs) with textual narratives. We probe them with respect to their common-sense knowledge of the structure and duration of events. We evaluate state-of-the-art LLMs on three tasks reflecting these abilities.
  arXiv  Detail & Related papers  (2023-11-14T18:57:15Z)
• Improving Factual Consistency of Text Summarization by Adversarially Decoupling Comprehension and Embellishment Abilities of LLMs [67.56087611675606]
  Large language models (LLMs) generate summaries that are factually inconsistent with original articles. These hallucinations are challenging to detect through traditional methods. We propose an adversarially DEcoupling method to disentangle the abilities of LLMs (DECENT)
  arXiv  Detail & Related papers  (2023-10-30T08:40:16Z)
• Are Large Language Models Really Robust to Word-Level Perturbations? [68.60618778027694]
  We propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools. Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions. Our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage.
  arXiv  Detail & Related papers  (2023-09-20T09:23:46Z)
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models [79.01926242857613]
  Large language models (LLMs) are prone to hallucinations, generating content that deviates from facts seen during pretraining. We propose a simple decoding strategy for reducing hallucinations with pretrained LLMs. We find that this Decoding by Contrasting Layers (DoLa) approach is able to better surface factual knowledge and reduce the generation of incorrect facts.
  arXiv  Detail & Related papers  (2023-09-07T17:45:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.