Teaching Language Models to Hallucinate Less with Synthetic Tasks
- URL: http://arxiv.org/abs/2310.06827v3
- Date: Tue, 7 Nov 2023 05:11:46 GMT
- Title: Teaching Language Models to Hallucinate Less with Synthetic Tasks
- Authors: Erik Jones, Hamid Palangi, Clarisse Sim\~oes, Varun Chandrasekaran,
Subhabrata Mukherjee, Arindam Mitra, Ahmed Awadallah, Ece Kamar
- Abstract summary: Large language models (LLMs) frequently hallucinate on abstractive summarization tasks.
We show that reducing hallucination on a synthetic task can also reduce hallucination on real-world downstream tasks.
- Score: 47.87453655902263
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large language models (LLMs) frequently hallucinate on abstractive
summarization tasks such as document-based question-answering, meeting
summarization, and clinical report generation, even though all necessary
information is included in context. However, optimizing LLMs to hallucinate
less on these tasks is challenging, as hallucination is hard to efficiently
evaluate at each optimization step. In this work, we show that reducing
hallucination on a synthetic task can also reduce hallucination on real-world
downstream tasks. Our method, SynTra, first designs a synthetic task where
hallucinations are easy to elicit and measure. It next optimizes the LLM's
system message via prefix-tuning on the synthetic task, and finally transfers
the system message to realistic, hard-to-optimize tasks. Across three realistic
abstractive summarization tasks, SynTra reduces hallucination for two
13B-parameter LLMs using only a synthetic retrieval task for supervision. We
also find that optimizing the system message rather than the model weights can
be critical; fine-tuning the entire model on the synthetic task can
counterintuitively increase hallucination. Overall, SynTra demonstrates that
the extra flexibility of working with synthetic data can help mitigate
undesired behaviors in practice.
Related papers
- ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models [65.12177400764506]
Large language models (LLMs) exhibit hallucinations in long-form question-answering tasks across various domains and wide applications.
Current hallucination detection and mitigation datasets are limited in domains and sizes.
This paper introduces an iterative self-training framework that simultaneously and progressively scales up the hallucination annotation dataset.
arXiv Detail & Related papers (2024-07-05T17:56:38Z) - Mitigating Object Hallucination via Data Augmented Contrastive Tuning [52.43197107069751]
Multimodal Large Language Models (MLLMs) tend to hallucinate factually inaccurate information.
We introduce a contrastive tuning method that can be applied to a pretrained off-the-shelf MLLM for mitigating hallucinations.
arXiv Detail & Related papers (2024-05-28T23:36:00Z) - Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback [48.065569871444275]
We propose detecting and mitigating hallucinations in Large Vision Language Models (LVLMs) via fine-grained AI feedback.
We generate a small-size hallucination annotation dataset by proprietary models.
Then, we propose a detect-then-rewrite pipeline to automatically construct preference dataset for training hallucination mitigating model.
arXiv Detail & Related papers (2024-04-22T14:46:10Z) - Hallucination Diversity-Aware Active Learning for Text Summarization [46.00645048690819]
Large Language Models (LLMs) have shown propensity to generate hallucinated outputs, i.e., texts that are factually incorrect or unsupported.
Existing methods for alleviating hallucinations typically require costly human annotations to identify and correct hallucinations in LLM outputs.
We propose the first active learning framework to alleviate LLM hallucinations, reducing costly human annotations of hallucination needed.
arXiv Detail & Related papers (2024-04-02T02:30:27Z) - Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models [0.0]
This paper takes a deep dive into Large Language Models (LLMs) behavior with respect to hallucinations.
It defines a token-level approach to identifying different kinds of hallucinations, and further utilizes this token-level tagging to improve the interpretability and faithfulness of LLMs.
arXiv Detail & Related papers (2023-12-22T00:31:46Z) - ORES: Open-vocabulary Responsible Visual Synthesis [104.7572323359984]
We formalize a new task, Open-vocabulary Responsible Visual Synthesis (ORES), where the synthesis model is able to avoid forbidden visual concepts.
To address this problem, we present a Two-stage Intervention (TIN) framework.
By introducing 1) rewriting with learnable instruction through a large-scale language model (LLM) and 2) synthesizing with prompt intervention on a diffusion model, it can effectively synthesize images avoiding any concepts but following the user's query as much as possible.
arXiv Detail & Related papers (2023-08-26T06:47:34Z) - ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real
Novel View Synthesis via Contrastive Learning [102.46382882098847]
We first investigate the effects of synthetic data in synthetic-to-real novel view synthesis.
We propose to introduce geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints.
Our method can render images with higher quality and better fine-grained details, outperforming existing generalizable novel view synthesis methods in terms of PSNR, SSIM, and LPIPS.
arXiv Detail & Related papers (2023-03-20T12:06:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.