What Evidence Do Language Models Find Convincing?
- URL: http://arxiv.org/abs/2402.11782v1
- Date: Mon, 19 Feb 2024 02:15:34 GMT
- Title: What Evidence Do Language Models Find Convincing?
- Authors: Alexander Wan, Eric Wallace, Dan Klein
- Abstract summary: We build a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts.
We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions.
Overall, we find that current models rely heavily on the relevance of a website to the query, while largely ignoring stylistic features that humans find important.
- Score: 103.67867531892988
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Retrieval-augmented language models are being increasingly tasked with
subjective, contentious, and conflicting queries such as "is aspartame linked
to cancer". To resolve these ambiguous queries, one must search through a large
range of websites and consider "which, if any, of this evidence do I find
convincing?". In this work, we study how LLMs answer this question. In
particular, we construct ConflictingQA, a dataset that pairs controversial
queries with a series of real-world evidence documents that contain different
facts (e.g., quantitative results), argument styles (e.g., appeals to
authority), and answers (Yes or No). We use this dataset to perform sensitivity
and counterfactual analyses to explore which text features most affect LLM
predictions. Overall, we find that current models rely heavily on the relevance
of a website to the query, while largely ignoring stylistic features that
humans find important such as whether a text contains scientific references or
is written with a neutral tone. Taken together, these results highlight the
importance of RAG corpus quality (e.g., the need to filter misinformation), and
possibly even a shift in how LLMs are trained to better align with human
judgements.
Related papers
- Attribute or Abstain: Large Language Models as Long Document Assistants [58.32043134560244]
We present a benchmark of 6 diverse long document tasks with attribution, and experiment with different approaches to attribution on 4 long documents.
We find that citation, i.e. response generation and evidence extraction in one step, mostly performs best.
We also find that evidence quality can predict response quality on datasets with simple responses, but not so for complex responses.
arXiv Detail & Related papers (2024-07-10T16:16:02Z) - Revealing Fine-Grained Values and Opinions in Large Language Models [42.48316407080442]
We analyse the responses of 156k large language models to 62 propositions of the Political Compass Test (PCT)
For fine-grained analysis, we propose to identify tropes in the responses: semantically similar phrases that are recurrent and consistent across different prompts.
We find that demographic features added to prompts significantly affect outcomes on the PCT, reflecting bias, as well as disparities between the results of tests when eliciting closed-form vs. open domain responses.
arXiv Detail & Related papers (2024-06-27T15:01:53Z) - LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing [106.45895712717612]
Large language models (LLMs) have shown remarkable versatility in various generative tasks.
This study focuses on the topic of LLMs assist NLP Researchers.
To our knowledge, this is the first work to provide such a comprehensive analysis.
arXiv Detail & Related papers (2024-06-24T01:30:22Z) - Investigating Annotator Bias in Large Language Models for Hate Speech Detection [6.25300580594605]
This paper delves into the biases present in Large Language Models (LLMs) when annotating hate speech data.
Specifically targeting highly vulnerable groups within these categories, we analyze annotator biases.
We introduce our custom hate speech detection dataset, HateSpeechCorpus, to conduct this research.
arXiv Detail & Related papers (2024-06-17T00:18:31Z) - Can't say cant? Measuring and Reasoning of Dark Jargons in Large Language Models [10.666290735480821]
This paper introduces a domain-specific Cant dataset and CantCounter evaluation framework.
Experiments reveal LLMs are susceptible to cant bypassing filters.
Updated models exhibit higher acceptance rates for cant queries.
arXiv Detail & Related papers (2024-04-25T17:25:53Z) - Customizing Language Model Responses with Contrastive In-Context Learning [7.342346948935483]
We propose an approach that uses contrastive examples to better describe our intent.
This involves providing positive examples that illustrate the true intent, along with negative examples that show what characteristics we want LLMs to avoid.
Before generating an answer, we ask the model to analyze the examples to teach itself what to avoid.
This reasoning step provides the model with the appropriate articulation of the user's need and guides it towards generting a better answer.
arXiv Detail & Related papers (2024-01-30T19:13:12Z) - You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments [37.03210795084276]
We examine whether the current format of prompting Large Language Models elicits responses in a consistent and robust manner.
Our experiments on 17 different LLMs reveal that even simple perturbations significantly downgrade a model's question-answering ability.
Our results suggest that the currently widespread practice of prompting is insufficient to accurately and reliably capture model perceptions.
arXiv Detail & Related papers (2023-11-16T09:50:53Z) - Do LLMs exhibit human-like response biases? A case study in survey
design [66.1850490474361]
We investigate the extent to which large language models (LLMs) reflect human response biases, if at all.
We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires.
Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
arXiv Detail & Related papers (2023-11-07T15:40:43Z) - Can Large Language Models Infer Causation from Correlation? [104.96351414570239]
We test the pure causal inference skills of large language models (LLMs)
We formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables.
We show that these models achieve almost close to random performance on the task.
arXiv Detail & Related papers (2023-06-09T12:09:15Z) - Competency Problems: On Finding and Removing Artifacts in Language Data [50.09608320112584]
We argue that for complex language understanding tasks, all simple feature correlations are spurious.
We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account.
arXiv Detail & Related papers (2021-04-17T21:34:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.