Learning Fine-Grained Grounded Citations for Attributed Large Language Models

• URL: http://arxiv.org/abs/2408.04568v1
• Date: Thu, 8 Aug 2024 16:28:22 GMT
• Title: Learning Fine-Grained Grounded Citations for Attributed Large Language Models
• Authors: Lei Huang, Xiaocheng Feng, Weitao Ma, Yuxuan Gu, Weihong Zhong, Xiachong Feng, Weijiang Yu, Weihua Peng, Duyu Tang, Dandan Tu, Bing Qin,
• Abstract summary: Front is a training framework designed to teach large language models (LLMs) to generate Fine-Grained Grounded Citations. Experiments on the ALCE benchmark demonstrate the efficacy of FRONT in generating superior grounded responses and highly supportive citations.
• Score: 44.79328335487421
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the impressive performance on information-seeking tasks, large language models (LLMs) still struggle with hallucinations. Attributed LLMs, which augment generated text with in-line citations, have shown potential in mitigating hallucinations and improving verifiability. However, current approaches suffer from suboptimal citation quality due to their reliance on in-context learning. Furthermore, the practice of citing only coarse document identifiers makes it challenging for users to perform fine-grained verification. In this work, we introduce FRONT, a training framework designed to teach LLMs to generate Fine-Grained Grounded Citations. By grounding model outputs in fine-grained supporting quotes, these quotes guide the generation of grounded and consistent responses, not only improving citation quality but also facilitating fine-grained verification. Experiments on the ALCE benchmark demonstrate the efficacy of FRONT in generating superior grounded responses and highly supportive citations. With LLaMA-2-7B, the framework significantly outperforms all the baselines, achieving an average of 14.21% improvement in citation quality across all datasets, even surpassing ChatGPT.

Related papers

• On the Capacity of Citation Generation by Large Language Models [38.47160164251295]
  Retrieval-augmented generation (RAG) appears as a promising method to alleviate the "hallucination" problem in large language models (LLMs)
  arXiv  Detail & Related papers  (2024-10-15T03:04:26Z)
• Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation [51.8188846284153]
  RAG has been widely adopted to enhance Large Language Models (LLMs) Attributed Text Generation (ATG) has attracted growing attention, which provides citations to support the model's responses in RAG. This paper proposes a fine-grained ATG method called ReClaim(Refer & Claim), which alternates the generation of references and answers step by step.
  arXiv  Detail & Related papers  (2024-07-01T20:47:47Z)
• ALiiCE: Evaluating Positional Fine-grained Citation Generation [54.19617927314975]
  We propose ALiiCE, the first automatic evaluation framework for fine-grained citation generation. Our framework first parses the sentence claim into atomic claims via dependency analysis and then calculates citation quality at the atomic claim level. We evaluate the positional fine-grained citation generation performance of several Large Language Models on two long-form QA datasets.
  arXiv  Detail & Related papers  (2024-06-19T09:16:14Z)
• Learning to Generate Answers with Citations via Factual Consistency Models [28.716998866121923]
  Large Language Models (LLMs) frequently hallucinate, impeding their reliability in mission-critical situations. This paper proposes a weakly-supervised fine-tuning method leveraging factual consistency models (FCMs) Focused learning is integrated into the objective, directing the fine-tuning process to emphasise the factual unit tokens.
  arXiv  Detail & Related papers  (2024-06-19T00:40:19Z)
• Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data [48.409306245463]
  We develop models that quote verbatim statements from trusted sources in pre-training data. Experimental results show that Quote-Tuning significantly increases verbatim quotes from high-quality pre-training documents by 55% to 130%.
  arXiv  Detail & Related papers  (2024-04-05T02:27:09Z)
• Training Language Models to Generate Text with Citations via Fine-grained Rewards [19.176465185343417]
  Large Language Models (LLMs) are prone to hallucination, and their responses often lack credibility due to missing references to reliable sources. We propose an effective training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations. On LLaMA-2-7B, the incorporation of fine-grained rewards achieves the best performance among the baselines, even surpassing that of GPT-3.5-turbo.
  arXiv  Detail & Related papers  (2024-02-06T19:00:40Z)
• CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation [87.44350003888646]
  Eval-Instruct can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting. CritiqueLLM is empirically shown to outperform ChatGPT and all the open-source baselines.
  arXiv  Detail & Related papers  (2023-11-30T16:52:42Z)
• Effective Large Language Model Adaptation for Improved Grounding and Citation Generation [48.07830615309543]
  This paper focuses on improving large language models (LLMs) by grounding their responses in retrieved passages and by providing citations. We propose a new framework, AGREE, that improves the grounding from a holistic perspective. Our framework tunes LLMs to selfground the claims in their responses and provide accurate citations to retrieved documents.
  arXiv  Detail & Related papers  (2023-11-16T03:22:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.