Probing Causes of Hallucinations in Neural Machine Translations
- URL: http://arxiv.org/abs/2206.12529v1
- Date: Sat, 25 Jun 2022 01:57:22 GMT
- Title: Probing Causes of Hallucinations in Neural Machine Translations
- Authors: Jianhao Yan, Fandong Meng, Jie Zhou
- Abstract summary: We propose to use probing methods to investigate the causes of hallucinations from the perspective of model architecture.
We find that hallucination is often accompanied by the deficient encoder, especially embeddings, and vulnerable cross-attentions.
- Score: 51.418245676894465
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Hallucination, one kind of pathological translations that bothers Neural
Machine Translation, has recently drawn much attention. In simple terms,
hallucinated translations are fluent sentences but barely related to source
inputs. Arguably, it remains an open problem how hallucination occurs. In this
paper, we propose to use probing methods to investigate the causes of
hallucinations from the perspective of model architecture, aiming to avoid such
problems in future architecture designs. By conducting experiments over various
NMT datasets, we find that hallucination is often accompanied by the deficient
encoder, especially embeddings, and vulnerable cross-attentions, while,
interestingly, cross-attention mitigates some errors caused by the encoder.
Related papers
- Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis [14.033320167387194]
A major challenge in their real-world application is hallucination, where LVLMs generate non-existent visual elements, eroding user trust.
We hypothesize that hidden factors, such as objects, contexts, and semantic foreground-background structures, induce hallucination.
By analyzing the causality between images, text prompts, and network saliency, we systematically explore interventions to block these factors.
arXiv Detail & Related papers (2024-12-04T01:23:57Z) - A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation [51.53917938874146]
We propose a possible solution for alleviating the hallucination in KGD by exploiting the dialogue-knowledge interaction.
Experimental results of our example implementation show that this method can reduce hallucination without disrupting other dialogue performance.
arXiv Detail & Related papers (2024-04-04T14:45:26Z) - Fine-grained Hallucination Detection and Editing for Language Models [109.56911670376932]
Large language models (LMs) are prone to generate factual errors, which are often called hallucinations.
We introduce a comprehensive taxonomy of hallucinations and argue that hallucinations manifest in diverse forms.
We propose a novel task of automatic fine-grained hallucination detection and construct a new evaluation benchmark, FavaBench.
arXiv Detail & Related papers (2024-01-12T19:02:48Z) - Hallucinations in Neural Automatic Speech Recognition: Identifying
Errors and Hallucinatory Models [11.492702369437785]
Hallucinations are semantically unrelated to the source utterance, yet still fluent and coherent.
We show that commonly used metrics, such as word error rates, cannot differentiate between hallucinatory and non-hallucinatory models.
We devise a framework for identifying hallucinations by analysing their semantic connection with the ground truth and their fluency.
arXiv Detail & Related papers (2024-01-03T06:56:56Z) - HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data [102.56792377624927]
hallucinations inherent in machine-generated data remain under-explored.
We present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm.
Our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA.
arXiv Detail & Related papers (2023-11-22T04:52:58Z) - Reducing Hallucinations in Neural Machine Translation with Feature
Attribution [54.46113444757899]
We present a case study focusing on model understanding and regularisation to reduce hallucinations in NMT.
We first use feature attribution methods to study the behaviour of an NMT model that produces hallucinations.
We then leverage these methods to propose a novel loss function that substantially helps reduce hallucinations and does not require retraining the model from scratch.
arXiv Detail & Related papers (2022-11-17T20:33:56Z) - Looking for a Needle in a Haystack: A Comprehensive Study of
Hallucinations in Neural Machine Translation [17.102338932907294]
We set foundations for the study of NMT hallucinations.
We propose DeHallucinator, a simple method for alleviating hallucinations at test time.
arXiv Detail & Related papers (2022-08-10T12:44:13Z) - VALHALLA: Visual Hallucination for Machine Translation [64.86515924691899]
We introduce a visual hallucination framework, called VALHALLA.
It requires only source sentences at inference time and instead uses hallucinated visual representations for multimodal machine translation.
In particular, given a source sentence an autoregressive hallucination transformer is used to predict a discrete visual representation from the input text.
arXiv Detail & Related papers (2022-05-31T20:25:15Z) - The Curious Case of Hallucinations in Neural Machine Translation [5.3180458405676205]
hallucinations in Neural Machine Translation lie at an extreme end on the spectrum of NMT pathologies.
We consider hallucinations under corpus-level noise (without any source perturbation) and demonstrate that two prominent types of natural hallucinations could be generated and explained through specific corpus-level noise patterns.
We elucidate the phenomenon of hallucination amplification in popular data-generation processes such as Backtranslation and sequence-level Knowledge Distillation.
arXiv Detail & Related papers (2021-04-14T08:09:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.