The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in
Open-domain Conversational Question Answering
- URL: http://arxiv.org/abs/2305.16519v1
- Date: Thu, 25 May 2023 22:54:13 GMT
- Title: The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in
Open-domain Conversational Question Answering
- Authors: Sabrina Chiesurin, Dimitris Dimakopoulos, Marco Antonio Sobrevilla
Cabezudo, Arash Eshghi, Ioannis Papaioannou, Verena Rieser, Ioannis Konstas
- Abstract summary: We show that task-based systems which exhibit certain advanced linguistic dialog behaviors, such as lexical alignment, are in fact preferred and trusted more.
Our results highlight the danger of systems that appear to be trustworthy by parroting user input while providing an unfaithful response.
- Score: 20.439568097395995
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Large language models are known to produce output which sounds fluent and
convincing, but is also often wrong, e.g. "unfaithful" with respect to a
rationale as retrieved from a knowledge base. In this paper, we show that
task-based systems which exhibit certain advanced linguistic dialog behaviors,
such as lexical alignment (repeating what the user said), are in fact preferred
and trusted more, whereas other phenomena, such as pronouns and ellipsis are
dis-preferred. We use open-domain question answering systems as our test-bed
for task based dialog generation and compare several open- and closed-book
models. Our results highlight the danger of systems that appear to be
trustworthy by parroting user input while providing an unfaithful response.
Related papers
- Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning [84.94709351266557]
We focus on the trustworthiness of language models with respect to retrieval augmentation.
We deem that retrieval-augmented language models have the inherent capabilities of supplying response according to both contextual and parametric knowledge.
Inspired by aligning language models with human preference, we take the first step towards aligning retrieval-augmented language models to a status where it responds relying merely on the external evidence.
arXiv Detail & Related papers (2024-10-22T09:25:21Z) - RELIC: Investigating Large Language Model Responses using Self-Consistency [58.63436505595177]
Large Language Models (LLMs) are notorious for blending fact with fiction and generating non-factual content, known as hallucinations.
We propose an interactive system that helps users gain insight into the reliability of the generated text.
arXiv Detail & Related papers (2023-11-28T14:55:52Z) - Improving Factual Consistency for Knowledge-Grounded Dialogue Systems
via Knowledge Enhancement and Alignment [77.56326872997407]
Pretrained language models (PLMs) based knowledge-grounded dialogue systems are prone to generate responses that are factually inconsistent with the provided knowledge source.
Inspired by previous work which identified that feed-forward networks (FFNs) within Transformers are responsible for factual knowledge expressions, we investigate two methods to efficiently improve the factual expression capability.
arXiv Detail & Related papers (2023-10-12T14:44:05Z) - PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded
Dialogue Systems [59.1250765143521]
Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities.
We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework.
We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
arXiv Detail & Related papers (2023-09-19T08:27:09Z) - PK-Chat: Pointer Network Guided Knowledge Driven Generative Dialogue
Model [79.64376762489164]
PK-Chat is a Pointer network guided generative dialogue model, incorporating a unified pretrained language model and a pointer network over knowledge graphs.
The words generated by PK-Chat in the dialogue are derived from the prediction of word lists and the direct prediction of the external knowledge graph knowledge.
Based on the PK-Chat, a dialogue system is built for academic scenarios in the case of geosciences.
arXiv Detail & Related papers (2023-04-02T18:23:13Z) - KPT: Keyword-guided Pre-training for Grounded Dialog Generation [82.68787152707455]
We propose KPT (guided Pre-Training), a novel self-supervised pre-training method for grounded dialog generation.
Specifically, we use a pre-trained language model to extract the most uncertain tokens in the dialog as keywords.
We conduct extensive experiments on various few-shot knowledge-grounded generation tasks, including grounding on dialog acts, knowledge graphs, persona descriptions, and Wikipedia passages.
arXiv Detail & Related papers (2022-12-04T04:05:01Z) - Grounding in social media: An approach to building a chit-chat dialogue
model [9.247397520986999]
Building open-domain dialogue systems capable of rich human-like conversational ability is one of the fundamental challenges in language generation.
Current work on knowledge-grounded dialogue generation primarily focuses on persona incorporation or searching a fact-based structured knowledge source such as Wikipedia.
Our method takes a broader and simpler approach, which aims to improve the raw conversation ability of the system by mimicking the human response behavior on social media.
arXiv Detail & Related papers (2022-06-12T09:01:57Z) - Saying No is An Art: Contextualized Fallback Responses for Unanswerable
Dialogue Queries [3.593955557310285]
Most dialogue systems rely on hybrid approaches for generating a set of ranked responses.
We design a neural approach which generates responses which are contextually aware with the user query.
Our simple approach makes use of rules over dependency parses and a text-to-text transformer fine-tuned on synthetic data of question-response pairs.
arXiv Detail & Related papers (2020-12-03T12:34:22Z) - NUANCED: Natural Utterance Annotation for Nuanced Conversation with
Estimated Distributions [36.00476428803116]
In this work, we attempt to build a user-centric dialogue system.
We first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions.
We build a new dataset named NUANCED that focuses on such realistic settings for conversational recommendation.
arXiv Detail & Related papers (2020-10-24T03:23:14Z) - Context-Sensitive Generation Network for Handing Unknown Slot Values in
Dialogue State Tracking [38.96838255645715]
We propose a novel Context-Sensitive Generation network (CSG) which can facilitate the representation of out-of-vocabulary words.
Our proposed method performs better than the state-of-the-art baselines.
arXiv Detail & Related papers (2020-05-08T09:22:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.