Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge
Selection
- URL: http://arxiv.org/abs/2308.15711v1
- Date: Wed, 30 Aug 2023 02:22:40 GMT
- Title: Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge
Selection
- Authors: Hongjin Qian, Zhicheng Dou, Jiejun Tan, Haonan Chen, Haoqi Gu, Ruofei
Lai, Xinyu Zhang, Zhao Cao, Ji-Rong Wen
- Abstract summary: Language models (LMs) have revolutionized the way we interact with information, but they often generate nonfactual text.
Previous methods use external knowledge as references for text generation to enhance factuality but often struggle with the knowledge mix-up of irrelevant references.
We present DKGen, which divide the text generation process into an iterative process.
- Score: 71.20871905457174
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Language models (LMs) have revolutionized the way we interact with
information, but they often generate nonfactual text, raising concerns about
their reliability. Previous methods use external knowledge as references for
text generation to enhance factuality but often struggle with the knowledge
mix-up(e.g., entity mismatch) of irrelevant references. Besides,as the length
of the output text grows, the randomness of sampling can escalate,
detrimentally impacting the factual accuracy of the generated text. In this
paper, we present DKGen, which divide the text generation process into an
iterative process. In each iteration, DKGen takes the input query, the
previously generated text and a subset of the reference passages as input to
generate short text. During the process, the subset is dynamically selected
from the full passage set based on their relevance to the previously generated
text and the query, largely eliminating the irrelevant references from input.
To further enhance DKGen's ability to correctly use these external knowledge,
DKGen distills the relevance order of reference passages to the cross-attention
distribution of decoder. We train and evaluate DKGen on a large-scale benchmark
dataset. Experiment results show that DKGen outperforms all baseline models.
Related papers
- Attribute First, then Generate: Locally-attributable Grounded Text Generation [33.371400233333326]
We introduce a locally-attributable text generation approach, prioritizing concise attributions.
Our method, named "Attribute First, then Generate", breaks down the conventional end-to-end generation process into three intuitive steps.
arXiv Detail & Related papers (2024-03-25T18:41:47Z) - RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder
for Language Modeling [79.56442336234221]
We introduce RegaVAE, a retrieval-augmented language model built upon the variational auto-encoder (VAE)
It encodes the text corpus into a latent space, capturing current and future information from both source and target text.
Experimental results on various datasets demonstrate significant improvements in text generation quality and hallucination removal.
arXiv Detail & Related papers (2023-10-16T16:42:01Z) - Text-To-KG Alignment: Comparing Current Methods on Classification Tasks [2.191505742658975]
knowledge graphs (KG) provide dense and structured representations of factual information.
Recent work has focused on creating pipeline models that retrieve information from KGs as additional context.
It is not known how current methods compare to a scenario where the aligned subgraph is completely relevant to the query.
arXiv Detail & Related papers (2023-06-05T13:45:45Z) - Evaluating Factual Consistency of Texts with Semantic Role Labeling [3.1776833268555134]
We introduce SRLScore, a reference-free evaluation metric designed with text summarization in mind.
A final factuality score is computed by an adjustable scoring mechanism.
Correlation with human judgments on English summarization datasets shows that SRLScore is competitive with state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T17:59:42Z) - Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation [92.1582872870226]
We propose a new grounded keys-to-text generation task.
The task is to generate a factual description about an entity given a set of guiding keys, and grounding passages.
Inspired by recent QA-based evaluation measures, we propose an automatic metric, MAFE, for factual correctness of generated descriptions.
arXiv Detail & Related papers (2022-12-04T23:59:41Z) - Towards Computationally Verifiable Semantic Grounding for Language
Models [18.887697890538455]
The paper conceptualizes the LM as a conditional model generating text given a desired semantic message formalized as a set of entity-relationship triples.
It embeds the LM in an auto-encoder by feeding its output to a semantic fluency whose output is in the same representation domain as the input message.
We show that our proposed approaches significantly improve on the greedy search baseline.
arXiv Detail & Related papers (2022-11-16T17:35:52Z) - Rethinking Text Line Recognition Models [57.47147190119394]
We consider two decoder families (Connectionist Temporal Classification and Transformer) and three encoder modules (Bidirectional LSTMs, Self-Attention, and GRCLs)
We compare their accuracy and performance on widely used public datasets of scene and handwritten text.
Unlike the more common Transformer-based models, this architecture can handle inputs of arbitrary length.
arXiv Detail & Related papers (2021-04-15T21:43:13Z) - POINTER: Constrained Progressive Text Generation via Insertion-based
Generative Pre-training [93.79766670391618]
We present POINTER, a novel insertion-based approach for hard-constrained text generation.
The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner.
The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
arXiv Detail & Related papers (2020-05-01T18:11:54Z) - Learning to Select Bi-Aspect Information for Document-Scale Text Content
Manipulation [50.01708049531156]
We focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer.
In detail, the input is a set of structured records and a reference text for describing another recordset.
The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference.
arXiv Detail & Related papers (2020-02-24T12:52:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.