Survey of Hallucination in Natural Language Generation

• URL: http://arxiv.org/abs/2202.03629v7
• Date: Sun, 14 Jul 2024 12:40:59 GMT
• Title: Survey of Hallucination in Natural Language Generation
• Authors: Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Delong Chen, Wenliang Dai, Ho Shu Chan, Andrea Madotto, Pascale Fung,
• Abstract summary: Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies. Deep learning based generation is prone to hallucinate unintended text, which degrades the system performance. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.
• Score: 69.9926849848132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation; and (3) hallucinations in large language models (LLMs). This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.

Related papers

• Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions [7.064953237013352]
  We focus on the research works that focus on text generation for visualizations. To characterize the NLG problem and the design space of proposed solutions, we pose five Wh-questions. We categorize the solutions used in the surveyed papers based on these "five Wh-questions"
  arXiv  Detail & Related papers  (2024-09-29T15:53:18Z)
• Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation [63.064204206220936]
  Foundational Large Language Models (LLMs) have changed the way we perceive technology. They have been shown to excel in tasks ranging from poem writing to coding to essay generation and puzzle solving. With the incorporation of image generation capability, they have become more comprehensive and versatile AI tools. Currently identified flaws include hallucination, biases, and bypassing restricted commands to generate harmful content.
  arXiv  Detail & Related papers  (2024-08-27T14:40:16Z)
• Analysis of Plan-based Retrieval for Grounded Text Generation [78.89478272104739]
  hallucinations occur when a language model is given a generation task outside its parametric knowledge. A common strategy to address this limitation is to infuse the language models with retrieval mechanisms. We analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations.
  arXiv  Detail & Related papers  (2024-08-20T02:19:35Z)
• Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation [15.67906403625006]
  The evaluation system for hallucination is complex and diverse, lacking clear organization. This survey aims to help researchers identify current limitations in hallucination evaluation and highlight future research directions.
  arXiv  Detail & Related papers  (2024-04-18T09:52:18Z)
• Cognitive Mirage: A Review of Hallucinations in Large Language Models [10.86850565303067]
  We present a novel taxonomy of hallucinations from various text generation tasks. We provide theoretical insights, detection methods and improvement approaches. As hallucinations garner significant attention, we will maintain updates on relevant research progress.
  arXiv  Detail & Related papers  (2023-09-13T08:33:09Z)
• Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods [48.47413103662829]
  Natural Language Generation (NLG) has made great progress in recent years due to the development of deep learning techniques such as pre-trained language models. However, the faithfulness problem that the generated text usually contains unfaithful or non-factual information has become the biggest challenge.
  arXiv  Detail & Related papers  (2022-03-10T08:28:32Z)
• A Survey on Retrieval-Augmented Text Generation [53.04991859796971]
  Retrieval-augmented text generation has remarkable advantages and has achieved state-of-the-art performance in many NLP tasks. It firstly highlights the generic paradigm of retrieval-augmented generation, and then it reviews notable approaches according to different tasks.
  arXiv  Detail & Related papers  (2022-02-02T16:18:41Z)
• Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey [54.34370423151014]
  This paper surveys the components of modeling approaches relaying task impacts across various generation tasks such as storytelling, summarization, translation etc. We present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them.
  arXiv  Detail & Related papers  (2020-10-14T17:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.