Converge to the Truth: Factual Error Correction via Iterative
Constrained Editing
- URL: http://arxiv.org/abs/2211.12130v1
- Date: Tue, 22 Nov 2022 10:03:13 GMT
- Title: Converge to the Truth: Factual Error Correction via Iterative
Constrained Editing
- Authors: Jiangjie Chen, Rui Xu, Wenxuan Zeng, Changzhi Sun, Lei Li, Yanghua
Xiao
- Abstract summary: We propose VENCE, a novel method for factual error correction (FEC) with minimal edits.
VENCE formulates the FEC problem as iterative sampling editing actions with respect to a target density function.
Experiments on a public dataset show that VENCE improves the well-adopted SARI metric by 5.3 (or a relative improvement of 11.8%) over the previous best distantly-supervised methods.
- Score: 30.740281040892086
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Given a possibly false claim sentence, how can we automatically correct it
with minimal editing? Existing methods either require a large number of pairs
of false and corrected claims for supervised training or do not handle well
errors spanning over multiple tokens within an utterance. In this paper, we
propose VENCE, a novel method for factual error correction (FEC) with minimal
edits. VENCE formulates the FEC problem as iterative sampling editing actions
with respect to a target density function. We carefully design the target
function with predicted truthfulness scores from an offline trained fact
verification model. VENCE samples the most probable editing positions based on
back-calculated gradients of the truthfulness score concerning input tokens and
the editing actions using a distantly-supervised language model (T5).
Experiments on a public dataset show that VENCE improves the well-adopted SARI
metric by 5.3 (or a relative improvement of 11.8%) over the previous best
distantly-supervised methods.
Related papers
- Improving Factual Error Correction by Learning to Inject Factual Errors [16.01718112369659]
Factual error correction aims to revise factual errors in false claims with minimal editing.
Existing methods typically adopt the mask-then-correct paradigm.
We propose a three-step distantly supervised method: mask-corrupt-correct.
arXiv Detail & Related papers (2023-12-12T08:02:06Z) - Lyra: Orchestrating Dual Correction in Automated Theorem Proving [65.48893396979807]
Lyra is a new framework that employs two distinct correction mechanisms: Tool Correction and Conjecture Correction.
Tool Correction contributes to mitigating hallucinations, thereby improving the overall accuracy of the proof.
Conjecture Correction refines generation with instruction but does not collect paired (generation, error & refinement) prompts.
arXiv Detail & Related papers (2023-09-27T17:29:41Z) - Chinese Spelling Correction as Rephrasing Language Model [63.65217759957206]
We study Chinese Spelling Correction (CSC), which aims to detect and correct the potential spelling errors in a given sentence.
Current state-of-the-art methods regard CSC as a sequence tagging task and fine-tune BERT-based models on sentence pairs.
We propose Rephrasing Language Model (ReLM), where the model is trained to rephrase the entire sentence by infilling additional slots, instead of character-to-character tagging.
arXiv Detail & Related papers (2023-08-17T06:04:28Z) - Reducing Sequence Length by Predicting Edit Operations with Large
Language Models [50.66922361766939]
This paper proposes predicting edit spans for the source text for local sequence transduction tasks.
We apply instruction tuning for Large Language Models on the supervision data of edit spans.
Experiments show that the proposed method achieves comparable performance to the baseline in four tasks.
arXiv Detail & Related papers (2023-05-19T17:51:05Z) - From Spelling to Grammar: A New Framework for Chinese Grammatical Error
Correction [12.170714706174314]
Chinese Grammatical Error Correction (CGEC) aims to generate a correct sentence from an erroneous sequence.
This paper divides the CGEC task into two steps, namely spelling error correction and grammatical error correction.
We propose a novel zero-shot approach for spelling error correction, which is simple but effective.
To handle grammatical error correction, we design part-of-speech features and semantic class features to enhance the neural network model.
arXiv Detail & Related papers (2022-11-03T07:30:09Z) - Memory-Based Model Editing at Scale [102.28475739907498]
Existing model editors struggle to accurately model an edit's intended scope.
We propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC)
SERAC stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed.
arXiv Detail & Related papers (2022-06-13T23:40:34Z) - Factual Error Correction for Abstractive Summaries Using Entity
Retrieval [57.01193722520597]
We propose an efficient factual error correction system RFEC based on entities retrieval post-editing process.
RFEC retrieves the evidence sentences from the original document by comparing the sentences with the target summary.
Next, RFEC detects the entity-level errors in the summaries by considering the evidence sentences and substitutes the wrong entities with the accurate entities from the evidence sentences.
arXiv Detail & Related papers (2022-04-18T11:35:02Z) - Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese
Grammatical Error Correction [49.25830718574892]
We present a new framework named Tail-to-Tail (textbfTtT) non-autoregressive sequence prediction.
Considering that most tokens are correct and can be conveyed directly from source to target, and the error positions can be estimated and corrected.
Experimental results on standard datasets, especially on the variable-length datasets, demonstrate the effectiveness of TtT in terms of sentence-level Accuracy, Precision, Recall, and F1-Measure.
arXiv Detail & Related papers (2021-06-03T05:56:57Z) - Evidence-based Factual Error Correction [18.52583883901634]
This paper introduces the task of factual error correction.
It provides a mechanism to correct written texts that are refuted or only partially supported by evidence.
arXiv Detail & Related papers (2021-06-02T11:00:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.