Related papers: Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar

Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar

URL: http://arxiv.org/abs/2311.09969v2
Date: Tue, 21 Nov 2023 09:49:25 GMT
Title: Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar
Authors: Celina Kacperski, Mona Bielig, Mykola Makhortykh, Maryna Sydorova, Roberto Ulloa
Abstract summary: This study examines whether confirmation-biased queries prompted into Google Scholar and Semantic Scholar will yield skewed results. Technology-related queries displaying more significant disparities. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence.
Score: 0.0
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Researchers rely on academic web search engines to find scientific sources, but search engine mechanisms may selectively present content that aligns with biases embedded in the queries. This study examines whether confirmation-biased queries prompted into Google Scholar and Semantic Scholar will yield skewed results. Six queries (topics across health and technology domains such as "vaccines" or "internet use") were analyzed for disparities in search results. We confirm that biased queries (targeting "benefits" or "risks") affect search results in line with the bias, with technology-related queries displaying more significant disparities. Overall, Semantic Scholar exhibited fewer disparities than Google Scholar. Topics rated as more polarizing did not consistently show more skewed results. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence. More research is needed to explore how scientific inquiry and academic search engines interact.

Related papers

Auditing Google's Search Algorithm: Measuring News Diversity Across Brazil, the UK, and the US [0.0]
This study examines the influence of Google's search algorithm on news diversity by analyzing search results in Brazil, the UK, and the US. It explores how Google's system preferentially favors a limited number of news outlets. Findings indicate a slight leftward bias in search outcomes and a preference for popular, often national outlets.
arXiv Detail & Related papers (2024-10-31T11:49:16Z)
Taxonomy-guided Semantic Indexing for Academic Paper Search [51.07749719327668]
TaxoIndex is a semantic index framework for academic paper search. It organizes key concepts from papers as a semantic index guided by an academic taxonomy. It can be flexibly employed to enhance existing dense retrievers.
arXiv Detail & Related papers (2024-10-25T00:00:17Z)
What Evidence Do Language Models Find Convincing? [94.90663008214918]
We build a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts. We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions. Overall, we find that current models rely heavily on the relevance of a website to the query, while largely ignoring stylistic features that humans find important.
arXiv Detail & Related papers (2024-02-19T02:15:34Z)
Navigating the Thin Line: Examining User Behavior in Search to Detect Engagement and Backfire Effects [0.0]
We investigate whether different levels of bias metrics and search results presentation can affect the stance diversity consumption and search behavior of opinionated users. Our results show that exposing participants to (counter-attitudinally) biased search results increases their consumption of attitude-opposing content. We also found that bias was associated with a trend toward overall fewer interactions within the search page.
arXiv Detail & Related papers (2024-01-20T10:28:25Z)
Algorithmic amplification of biases on Google Search [0.6167267484484484]
This paper investigates how individuals' preexisting attitudes influence the modern information-seeking process. Individuals with opposing attitudes on abortion receive different search results. Google Search engine reinforces preexisting beliefs in search results.
arXiv Detail & Related papers (2024-01-17T08:24:57Z)
User Attitudes to Content Moderation in Web Search [49.1574468325115]
We examine the levels of support for different moderation practices applied to potentially misleading and/or potentially offensive content in web search. We find that the most supported practice is informing users about potentially misleading or offensive content, and the least supported one is the complete removal of search results. More conservative users and users with lower levels of trust in web search results are more likely to be against content moderation in web search.
arXiv Detail & Related papers (2023-10-05T10:57:15Z)
Explaining Search Result Stances to Opinionated People [0.0]
We investigate whether stance labels and their explanations can help users consume more diverse search results. We find that stance labels and explanations lead to a more diverse search result consumption. However, we do not find evidence for systematic opinion change among users in this context.
arXiv Detail & Related papers (2023-09-15T15:08:24Z)
Evaluating Verifiability in Generative Search Engines [70.59477647085387]
Generative search engines directly generate responses to user queries, along with in-line citations. We conduct human evaluation to audit four popular generative search engines. We find that responses from existing generative search engines are fluent and appear informative, but frequently contain unsupported statements and inaccurate citations.
arXiv Detail & Related papers (2023-04-19T17:56:12Z)
A Measure of Research Taste [91.3755431537592]
We present a citation-based measure that rewards both productivity and taste. The presented measure, CAP, balances the impact of publications and their quantity. We analyze the characteristics of CAP for highly-cited researchers in biology, computer science, economics, and physics.
arXiv Detail & Related papers (2021-05-17T18:01:47Z)
The Matter of Chance: Auditing Web Search Results Related to the 2020 U.S. Presidential Primary Elections Across Six Search Engines [68.8204255655161]
We look at the text search results for "us elections", "donald trump", "joe biden" and "bernie sanders" queries on Google, Baidu, Bing, DuckDuckGo, Yahoo, and Yandex. Our findings indicate substantial differences in the search results between search engines and multiple discrepancies within the results generated for different agents.
arXiv Detail & Related papers (2021-05-03T11:18:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.