In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations

• URL: http://arxiv.org/abs/2602.15456v1
• Date: Tue, 17 Feb 2026 09:45:22 GMT
• Title: In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
• Authors: Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander,
• Abstract summary: Large Language Models (LLMs) are increasingly being deployed as interfaces to information on online platforms.<n>LLMs govern the information users receive, by drawing users' attention to particular instances of retrieved information at the expense of others.<n>We find that several models consistently exhibit strong and predictable source preferences.
• Score: 19.98336514529218
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agents based on Large Language Models (LLMs) are increasingly being deployed as interfaces to information on online platforms. These agents filter, prioritize, and synthesize information retrieved from the platforms' back-end databases or via web search. In these scenarios, LLM agents govern the information users receive, by drawing users' attention to particular instances of retrieved information at the expense of others. While much prior work has focused on biases in the information LLMs themselves generate, less attention has been paid to the factors that influence what information LLMs select and present to users. We hypothesize that when information is attributed to specific sources (e.g., particular publishers, journals, or platforms), current LLMs exhibit systematic latent source preferences- that is, they prioritize information from some sources over others. Through controlled experiments on twelve LLMs from six model providers, spanning both synthetic and real-world tasks, we find that several models consistently exhibit strong and predictable source preferences. These preferences are sensitive to contextual framing, can outweigh the influence of content itself, and persist despite explicit prompting to avoid them. They also help explain phenomena such as the observed left-leaning skew in news recommendations in prior work. Our findings advocate for deeper investigation into the origins of these preferences, as well as for mechanisms that provide users with transparency and control over the biases guiding LLM-powered agents.

Related papers

• Whose Facts Win? LLM Source Preferences under Knowledge Conflicts [4.587118047944915]
  We investigate how source preferences affect large language models (LLMs) resolution of inter-context knowledge conflicts in English.<n>We find that LLMs prefer institutionally-corroborated information over information from people and social media.<n>These source preferences can be reversed by simply repeating information from less credible sources.
  arXiv  Detail & Related papers  (2026-01-07T09:35:35Z)
• Are Large Language Models Sensitive to the Motives Behind Communication? [9.246336669308665]
  Large language models (LLMs) and AI agents process information inherently framed by humans' intentions and incentives.<n>For LLMs to be effective in the real world, they too must critically evaluate content by factoring in the motivations of the source.<n>We employ controlled experiments from cognitive science to verify that LLMs' behavior is consistent with rational models of learning from motivated testimony.<n>We find that LLMs' inferences do not track the rational models nearly as closely -- partly due to additional information that distracts them from vigilance-relevant considerations.
  arXiv  Detail & Related papers  (2025-10-22T15:35:00Z)
• How Do LLM-Generated Texts Impact Term-Based Retrieval Models? [76.92519309816008]
  This paper investigates the influence of large language models (LLMs) on term-based retrieval models.<n>Our linguistic analysis reveals that LLM-generated texts exhibit smoother high-frequency and steeper low-frequency Zipf slopes.<n>Our study further explores whether term-based retrieval models demonstrate source bias, concluding that these models prioritize documents whose term distributions closely correspond to those of the queries.
  arXiv  Detail & Related papers  (2025-08-25T06:43:27Z)
• Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment [49.81946749379338]
  This work seeks to analyze the capacity of Transformers-based systems to learn demographic biases present in the data.<n>We propose a privacy-enhancing framework to reduce gender information from the learning pipeline as a way to mitigate biased behaviors in the final tools.
  arXiv  Detail & Related papers  (2025-06-13T15:29:43Z)
• LLM Enhancer: Merged Approach using Vector Embedding for Reducing Large Language Model Hallucinations with External Knowledge [0.0]
  Large Language Models (LLMs) have demonstrated the capability to generate human like, natural responses across a range of tasks.<n>This paper introduces the LLM ENHANCER system, designed to integrate multiple online sources such as Google, Wikipedia, and DuckDuckGo to enhance data accuracy.
  arXiv  Detail & Related papers  (2025-04-29T19:27:04Z)
• How LLMs Fail to Support Fact-Checking [4.918358353535447]
  Large Language Models (LLMs) can amplify online misinformation, but show promise in tackling misinformation.<n>We empirically study the capabilities of three LLMs -- ChatGPT, Gemini, and Claude -- in countering political misinformation.<n>Our findings suggest that models struggle to ground their responses in real news sources, and tend to prefer citing left-leaning sources.
  arXiv  Detail & Related papers  (2025-02-28T07:12:03Z)
• The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generations [40.498553309980764]
  We study the interplay between verifiability and utility of information-sharing tools. We find that users prefer search engines over large language models for high-stakes queries.
  arXiv  Detail & Related papers  (2024-11-26T12:34:52Z)
• PRISM: A Methodology for Auditing Biases in Large Language Models [9.751718230639376]
  PRISM is a flexible, inquiry-based methodology for auditing Large Language Models. It seeks to illicit such positions indirectly through task-based inquiry prompting rather than direct inquiry of said preferences.
  arXiv  Detail & Related papers  (2024-10-24T16:57:20Z)
• Cognitive Biases in Large Language Models for News Recommendation [68.90354828533535]
  This paper explores the potential impact of cognitive biases on large language models (LLMs) based news recommender systems. We discuss strategies to mitigate these biases through data augmentation, prompt engineering and learning algorithms aspects.
  arXiv  Detail & Related papers  (2024-10-03T18:42:07Z)
• LLM-PBE: Assessing Data Privacy in Large Language Models [111.58198436835036]
  Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs.
  arXiv  Detail & Related papers  (2024-08-23T01:37:29Z)
• CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models [60.59638232596912]
  We introduce CLAMBER, a benchmark for evaluating large language models (LLMs) Building upon the taxonomy, we construct 12K high-quality data to assess the strengths, weaknesses, and potential risks of various off-the-shelf LLMs. Our findings indicate the limited practical utility of current LLMs in identifying and clarifying ambiguous user queries.
  arXiv  Detail & Related papers  (2024-05-20T14:34:01Z)
• On the Risk of Misinformation Pollution with Large Language Models [127.1107824751703]
  We investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation. Our study reveals that LLMs can act as effective misinformation generators, leading to a significant degradation in the performance of Open-Domain Question Answering (ODQA) systems.
  arXiv  Detail & Related papers  (2023-05-23T04:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.