Trustworthiness Evaluations of Search Results: The Impact of Rank and Misinformation

• URL: http://arxiv.org/abs/2309.11029v1
• Date: Wed, 20 Sep 2023 03:19:11 GMT
• Title: Trustworthiness Evaluations of Search Results: The Impact of Rank and Misinformation
• Abstract summary: We show that although higher-ranked results are clicked more often, they are not more trusted. We also show that misinformation did not change trust in accurate results below it. This work addresses concerns about how people evaluate information in search, and illustrates the dangers of generic prevention approaches.
• Score: 5.847084649531299
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Users rely on search engines for information in critical contexts, such as public health emergencies. Understanding how users evaluate the trustworthiness of search results is therefore essential. Research has identified rank and the presence of misinformation as factors impacting perceptions and click behavior in search. Here, we elaborate on these findings by measuring the effects of rank and misinformation, as well as warning banners, on the perceived trustworthiness of individual results in search. We conducted three online experiments (N=3196) using Covid-19-related queries to address this question. We show that although higher-ranked results are clicked more often, they are not more trusted. We also show that misinformation did not change trust in accurate results below it. However, a warning about unreliable sources backfired, decreasing trust in accurate information but not misinformation. This work addresses concerns about how people evaluate information in search, and illustrates the dangers of generic prevention approaches.

Related papers

• From Verification to Amplification: Auditing Reverse Image Search as Algorithmic Gatekeeping in Visual Misinformation Fact-checking [6.612119513224063]
  This study systematically audits Google RIS by reversely searching newly identified misleading images over a 15-day window.<n>We find that Google RIS returns a substantial volume of irrelevant information and repeated misinformation, whereas debunking content constitutes less than 30% of search results.
  arXiv  Detail & Related papers  (2026-03-10T03:07:27Z)
• A Causal Information-Flow Framework for Unbiased Learning-to-Rank [52.54102347581931]
  In web search and recommendation systems, user clicks are widely used to train ranking models.<n>We introduce a novel causal learning-based ranking framework that extends Unbiased Learning-to-Rank.<n>Our method consistently reduces measured bias leakage and improves ranking performance.
  arXiv  Detail & Related papers  (2026-01-09T07:19:35Z)
• Data Fusion for Partial Identification of Causal Effects [62.56890808004615]
  We propose a novel partial identification framework that enables researchers to answer key questions.<n>Is the causal effect positive or negative? and How severe must assumption violations be to overturn this conclusion?<n>We apply our framework to the Project STAR study, which investigates the effect of classroom size on students' third-grade standardized test performance.
  arXiv  Detail & Related papers  (2025-05-30T07:13:01Z)
• Query Smarter, Trust Better? Exploring Search Behaviours for Verifying News Accuracy [35.07647423247397]
  This study explores how different query generation strategies affect news verification and whether the way people search influences the accuracy of their information evaluation. The results show that search behaviour significantly affects trust in news, with successful searches involving multiple queries yielding higher-quality results. Although 'Boost' interventions had limited impact, the study suggests that interface design encouraging users to thoroughly review search results can enhance query formulation.
  arXiv  Detail & Related papers  (2025-04-07T14:50:13Z)
• MisinfoEval: Generative AI in the Era of "Alternative Facts" [50.069577397751175]
  We introduce a framework for generating and evaluating large language model (LLM) based misinformation interventions. We present (1) an experiment with a simulated social media environment to measure effectiveness of misinformation interventions, and (2) a second experiment with personalized explanations tailored to the demographics and beliefs of users. Our findings confirm that LLM-based interventions are highly effective at correcting user behavior.
  arXiv  Detail & Related papers  (2024-10-13T18:16:50Z)
• "I don't trust them": Exploring Perceptions of Fact-checking Entities for Flagging Online Misinformation [3.6754294738197264]
  We conducted an online study with 655 US participants to explore user perceptions of eight categories of fact-checking entities across two misinformation topics. Our results hint at the need for further exploring fact-checking entities that may be perceived as neutral, as well as the potential for incorporating multiple assessments in such labels.
  arXiv  Detail & Related papers  (2024-10-01T17:01:09Z)
• Navigating the Thin Line: Examining User Behavior in Search to Detect Engagement and Backfire Effects [0.0]
  We investigate whether different levels of bias metrics and search results presentation can affect the stance diversity consumption and search behavior of opinionated users. Our results show that exposing participants to (counter-attitudinally) biased search results increases their consumption of attitude-opposing content. We also found that bias was associated with a trend toward overall fewer interactions within the search page.
  arXiv  Detail & Related papers  (2024-01-20T10:28:25Z)
• User Attitudes to Content Moderation in Web Search [49.1574468325115]
  We examine the levels of support for different moderation practices applied to potentially misleading and/or potentially offensive content in web search. We find that the most supported practice is informing users about potentially misleading or offensive content, and the least supported one is the complete removal of search results. More conservative users and users with lower levels of trust in web search results are more likely to be against content moderation in web search.
  arXiv  Detail & Related papers  (2023-10-05T10:57:15Z)
• Measuring the Effect of Influential Messages on Varying Personas [67.1149173905004]
  We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona might have upon seeing a news message. The proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response. This enables more accurate and comprehensive inference on the mental state of the persona.
  arXiv  Detail & Related papers  (2023-05-25T21:01:00Z)
• Approaches to Identify Vulnerabilities to Misinformation: A Research Agenda [24.80321868079655]
  Given the prevalence of online misinformation, Internet users have been shown to frequently fall victim to such information. We highlight two ongoing avenues of research to identify vulnerable users: detecting cognitive biases and exploring misinformation spreaders.
  arXiv  Detail & Related papers  (2022-10-21T00:33:39Z)
• Valid Inference After Causal Discovery [73.87055989355737]
  We develop tools for valid post-causal-discovery inference. We show that a naive combination of causal discovery and subsequent inference algorithms leads to highly inflated miscoverage rates.
  arXiv  Detail & Related papers  (2022-08-11T17:40:45Z)
• Misinfo Belief Frames: A Case Study on Covid & Climate News [49.979419711713795]
  We propose a formalism for understanding how readers perceive the reliability of news and the impact of misinformation. We introduce the Misinfo Belief Frames (MBF) corpus, a dataset of 66k inferences over 23.5k headlines. Our results using large-scale language modeling to predict misinformation frames show that machine-generated inferences can influence readers' trust in news headlines.
  arXiv  Detail & Related papers  (2021-04-18T09:50:11Z)
• Drink Bleach or Do What Now? Covid-HeRA: A Study of Risk-Informed Health Decision Making in the Presence of COVID-19 Misinformation [23.449057978351945]
  We frame health misinformation as a risk assessment task. We study the severity of each misinformation story and how readers perceive this severity. We evaluate several traditional and state-of-the-art models and show there is a significant gap in performance.
  arXiv  Detail & Related papers  (2020-10-17T08:34:57Z)
• The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively? [17.288917654501265]
  We study whether crowdsourcing is an effective and reliable method to assess statements truthfulness during a pandemic. We specifically target statements related to the COVID-19 health emergency, that is still ongoing at the time of the study. In our experiment, crowd workers are asked to assess the truthfulness of statements, as well as to provide evidence for the assessments as a URL and a text justification.
  arXiv  Detail & Related papers  (2020-08-13T05:53:24Z)
• Information Consumption and Social Response in a Segregated Environment: the Case of Gab [74.5095691235917]
  This work provides a characterization of the interaction patterns within Gab around the COVID-19 topic. We find that there are no strong statistical differences in the social response to questionable and reliable content. Our results provide insights toward the understanding of coordinated inauthentic behavior and on the early-warning of information operation.
  arXiv  Detail & Related papers  (2020-06-03T11:34:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.