Computational Fact-Checking of Online Discourse: Scoring scientific accuracy in climate change related news articles

• URL: http://arxiv.org/abs/2505.07409v1
• Date: Mon, 12 May 2025 10:03:15 GMT
• Title: Computational Fact-Checking of Online Discourse: Scoring scientific accuracy in climate change related news articles
• Authors: Tim Wittenborg, Constantin Sebastian Tremel, Markus Stocker, Sören Auer,
• Abstract summary: This work aims to quantify scientific accuracy of online media.<n>By semantifying media of unknown veracity, their statements can be compared against equally processed trusted sources.
• Score: 0.14999444543328289
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Democratic societies need reliable information. Misinformation in popular media such as news articles or videos threatens to impair civic discourse. Citizens are, unfortunately, not equipped to verify this content flood consumed daily at increasing rates. This work aims to semi-automatically quantify scientific accuracy of online media. By semantifying media of unknown veracity, their statements can be compared against equally processed trusted sources. We implemented a workflow using LLM-based statement extraction and knowledge graph analysis. Our neurosymbolic system was able to evidently streamline state-of-the-art veracity quantification. Evaluated via expert interviews and a user survey, the tool provides a beneficial veracity indication. This indicator, however, is unable to annotate public media at the required granularity and scale. Further work towards a FAIR (Findable, Accessible, Interoperable, Reusable) ground truth and complementary metrics are required to scientifically support civic discourse.

Related papers

• Profiling News Media for Factuality and Bias Using LLMs and the Fact-Checking Methodology of Human Experts [29.95198868148809]
  We propose a novel methodology that emulates the criteria that professional fact-checkers use to assess the factuality and political bias of an entire outlet.<n>We provide an in-depth error analysis of the effect of media popularity and region on model performance.
  arXiv  Detail & Related papers  (2025-06-14T15:49:20Z)
• Fact or Fiction? Can LLMs be Reliable Annotators for Political Truths? [2.321323878201932]
  Political misinformation poses challenges to democratic processes, shaping public opinion and trust in media. This study investigates the use of state-of-the-art large language models (LLMs) as reliable annotators for detecting political factuality in news articles.
  arXiv  Detail & Related papers  (2024-11-08T18:36:33Z)
• Mapping the Media Landscape: Predicting Factual Reporting and Political Bias Through Web Interactions [0.7249731529275342]
  We propose an extension to a recently presented news media reliability estimation method. We assess the classification performance of four reinforcement learning strategies on a large news media hyperlink graph. Our experiments, targeting two challenging bias descriptors, factual reporting and political bias, showed a significant performance improvement at the source media level.
  arXiv  Detail & Related papers  (2024-10-23T08:18:26Z)
• Missci: Reconstructing Fallacies in Misrepresented Science [84.32990746227385]
  Health-related misinformation on social networks can lead to poor decision-making and real-world dangers. Missci is a novel argumentation theoretical model for fallacious reasoning. We present Missci as a dataset to test the critical reasoning abilities of large language models.
  arXiv  Detail & Related papers  (2024-06-05T12:11:10Z)
• Correcting misinformation on social media with a large language model [14.69780455372507]
  Real-world misinformation, often multimodal, can be misleading using diverse tactics like conflating correlation with causation. Such misinformation is severely understudied, challenging to address, and harms various social domains, particularly on social media. We propose MUSE, an LLM augmented with access to and credibility evaluation of up-to-date information.
  arXiv  Detail & Related papers  (2024-03-17T10:59:09Z)
• Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection [50.07850264495737]
  "Prompt-and-Align" (P&A) is a novel prompt-based paradigm for few-shot fake news detection. We show that P&A sets new states-of-the-art for few-shot fake news detection performance by significant margins.
  arXiv  Detail & Related papers  (2023-09-28T13:19:43Z)
• ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media [74.93847489218008]
  We present a novel task, identifying manipulation of news on social media, which aims to detect manipulation in social media posts and identify manipulated or inserted information.<n>To study this task, we have proposed a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles.<n>Our analysis demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance.
  arXiv  Detail & Related papers  (2023-05-23T16:40:07Z)
• Unveiling the Hidden Agenda: Biases in News Reporting and Consumption [59.55900146668931]
  We build a six-year dataset on the Italian vaccine debate and adopt a Bayesian latent space model to identify narrative and selection biases. We found a nonlinear relationship between biases and engagement, with higher engagement for extreme positions. Analysis of news consumption on Twitter reveals common audiences among news outlets with similar ideological positions.
  arXiv  Detail & Related papers  (2023-01-14T18:58:42Z)
• Stance Detection with BERT Embeddings for Credibility Analysis of Information on Social Media [1.7616042687330642]
  We propose a model for detecting fake news using stance as one of the features along with the content of the article. Our work interprets the content with automatic feature extraction and the relevance of the text pieces. The experiment conducted on the real-world dataset indicates that our model outperforms the previous work and enables fake news detection with an accuracy of 95.32%.
  arXiv  Detail & Related papers  (2021-05-21T10:46:43Z)
• Machine Learning Explanations to Prevent Overtrust in Fake News Detection [64.46876057393703]
  This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news. We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms. For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
  arXiv  Detail & Related papers  (2020-07-24T05:42:29Z)
• Political audience diversity and news reliability in algorithmic ranking [54.23273310155137]
  We propose using the political diversity of a website's audience as a quality signal. Using news source reliability ratings from domain experts and web browsing data from a diverse sample of 6,890 U.S. citizens, we first show that websites with more extreme and less politically diverse audiences have lower journalistic standards.
  arXiv  Detail & Related papers  (2020-07-16T02:13:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.