Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar

• URL: http://arxiv.org/abs/2311.09969v2
• Date: Tue, 21 Nov 2023 09:49:25 GMT
• Title: Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar
• Authors: Celina Kacperski, Mona Bielig, Mykola Makhortykh, Maryna Sydorova, Roberto Ulloa
• Abstract summary: This study examines whether confirmation-biased queries prompted into Google Scholar and Semantic Scholar will yield skewed results. Technology-related queries displaying more significant disparities. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Researchers rely on academic web search engines to find scientific sources, but search engine mechanisms may selectively present content that aligns with biases embedded in the queries. This study examines whether confirmation-biased queries prompted into Google Scholar and Semantic Scholar will yield skewed results. Six queries (topics across health and technology domains such as "vaccines" or "internet use") were analyzed for disparities in search results. We confirm that biased queries (targeting "benefits" or "risks") affect search results in line with the bias, with technology-related queries displaying more significant disparities. Overall, Semantic Scholar exhibited fewer disparities than Google Scholar. Topics rated as more polarizing did not consistently show more skewed results. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence. More research is needed to explore how scientific inquiry and academic search engines interact.

Related papers

• What Evidence Do Language Models Find Convincing? [103.67867531892988]
  We build a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts. We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions. Overall, we find that current models rely heavily on the relevance of a website to the query, while largely ignoring stylistic features that humans find important.
  arXiv  Detail & Related papers  (2024-02-19T02:15:34Z)
• Navigating the Thin Line: Examining User Behavior in Search to Detect Engagement and Backfire Effects [0.0]
  We investigate whether different levels of bias metrics and search results presentation can affect the stance diversity consumption and search behavior of opinionated users. Our results show that exposing participants to (counter-attitudinally) biased search results increases their consumption of attitude-opposing content. We also found that bias was associated with a trend toward overall fewer interactions within the search page.
  arXiv  Detail & Related papers  (2024-01-20T10:28:25Z)
• Algorithmic amplification of biases on Google Search [0.6167267484484484]
  This paper investigates how individuals' preexisting attitudes influence the modern information-seeking process. Individuals with opposing attitudes on abortion receive different search results. Google Search engine reinforces preexisting beliefs in search results.
  arXiv  Detail & Related papers  (2024-01-17T08:24:57Z)
• User Attitudes to Content Moderation in Web Search [49.1574468325115]
  We examine the levels of support for different moderation practices applied to potentially misleading and/or potentially offensive content in web search. We find that the most supported practice is informing users about potentially misleading or offensive content, and the least supported one is the complete removal of search results. More conservative users and users with lower levels of trust in web search results are more likely to be against content moderation in web search.
  arXiv  Detail & Related papers  (2023-10-05T10:57:15Z)
• Explaining Search Result Stances to Opinionated People [0.0]
  We investigate whether stance labels and their explanations can help users consume more diverse search results. We find that stance labels and explanations lead to a more diverse search result consumption. However, we do not find evidence for systematic opinion change among users in this context.
  arXiv  Detail & Related papers  (2023-09-15T15:08:24Z)
• Evaluating Verifiability in Generative Search Engines [70.59477647085387]
  Generative search engines directly generate responses to user queries, along with in-line citations. We conduct human evaluation to audit four popular generative search engines. We find that responses from existing generative search engines are fluent and appear informative, but frequently contain unsupported statements and inaccurate citations.
  arXiv  Detail & Related papers  (2023-04-19T17:56:12Z)
• Award rate inequities in biomedical research [55.850540873687386]
  The authors performed an analysis of 14,263 biomedical research proposals with proposed start dates between 2010-2022 from the University of Michigan Medical School. There is a clear relationship between race/ethnicity and rates of proposal award. Black/African American and Asian researchers appear disadvantaged across all submission categories relative to White researchers.
  arXiv  Detail & Related papers  (2022-06-14T14:05:39Z)
• A Measure of Research Taste [91.3755431537592]
  We present a citation-based measure that rewards both productivity and taste. The presented measure, CAP, balances the impact of publications and their quantity. We analyze the characteristics of CAP for highly-cited researchers in biology, computer science, economics, and physics.
  arXiv  Detail & Related papers  (2021-05-17T18:01:47Z)
• The Matter of Chance: Auditing Web Search Results Related to the 2020 U.S. Presidential Primary Elections Across Six Search Engines [68.8204255655161]
  We look at the text search results for "us elections", "donald trump", "joe biden" and "bernie sanders" queries on Google, Baidu, Bing, DuckDuckGo, Yahoo, and Yandex. Our findings indicate substantial differences in the search results between search engines and multiple discrepancies within the results generated for different agents.
  arXiv  Detail & Related papers  (2021-05-03T11:18:19Z)
• Search Engine Similarity Analysis: A Combined Content and Rankings Approach [6.69087470775851]
  We present an analysis of the affinity of the two major search engines, Google and Bing, along with DuckDuckGo. We developed a new similarity metric that leverages both the content and the ranking of search responses. We found that Google stands apart, but Bing and DuckDuckGo are largely indistinguishable from each other.
  arXiv  Detail & Related papers  (2020-11-01T23:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.