From Toxicity in Online Comments to Incivility in American News: Proceed with Caution

• URL: http://arxiv.org/abs/2102.03671v1
• Date: Sat, 6 Feb 2021 21:49:17 GMT
• Title: From Toxicity in Online Comments to Incivility in American News: Proceed with Caution
• Authors: Anushree Hede, Oshin Agarwal, Linda Lu, Diana C. Mutz, Ani Nenkova
• Abstract summary: We test the Jigsaw Perspective API for its ability to detect the degree of incivility on a corpus that we developed. We demonstrate that toxicity models, as exemplified by Perspective, are inadequate for the analysis of incivility in news.
• Score: 6.770249285373613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to quantify incivility online, in news and in congressional debates, is of great interest to political scientists. Computational tools for detecting online incivility for English are now fairly accessible and potentially could be applied more broadly. We test the Jigsaw Perspective API for its ability to detect the degree of incivility on a corpus that we developed, consisting of manual annotations of civility in American news. We demonstrate that toxicity models, as exemplified by Perspective, are inadequate for the analysis of incivility in news. We carry out error analysis that points to the need to develop methods to remove spurious correlations between words often mentioned in the news, especially identity descriptors and incivility. Without such improvements, applying Perspective or similar models on news is likely to lead to wrong conclusions, that are not aligned with the human perception of incivility.

Related papers

• DocNet: Semantic Structure in Inductive Bias Detection Models [0.4779196219827508]
  In this paper, we explore an often overlooked aspect of bias detection in documents: the semantic structure of news articles. We present DocNet, a novel, inductive, and low-resource document embedding and bias detection model. We also demonstrate that the semantic structure of news articles from opposing partisan sides, as represented in document-level graph embeddings, have significant similarities.
  arXiv  Detail & Related papers  (2024-06-16T14:51:12Z)
• Hate Cannot Drive out Hate: Forecasting Conversation Incivility following Replies to Hate Speech [12.274054522085107]
  We propose a metric to measure conversation incivility based on the number of civil and uncivil comments. A linguistic analysis uncovers the differences in the language of replies that elicit follow-up conversations with high and low incivility.
  arXiv  Detail & Related papers  (2023-12-08T02:39:17Z)
• Adapting Fake News Detection to the Era of Large Language Models [48.5847914481222]
  We study the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news. Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa.
  arXiv  Detail & Related papers  (2023-11-02T08:39:45Z)
• Detecting Multidimensional Political Incivility on Social Media [2.3704813250344436]
  We present state-of-the-art incivility detection results using a large dataset of 13K political tweets. We observe that political incivility demonstrates a highly skewed distribution over users, and examine social factors that correlate with incivility at subpopulation and user-level.
  arXiv  Detail & Related papers  (2023-05-24T09:57:12Z)
• Unveiling the Hidden Agenda: Biases in News Reporting and Consumption [59.55900146668931]
  We build a six-year dataset on the Italian vaccine debate and adopt a Bayesian latent space model to identify narrative and selection biases. We found a nonlinear relationship between biases and engagement, with higher engagement for extreme positions. Analysis of news consumption on Twitter reveals common audiences among news outlets with similar ideological positions.
  arXiv  Detail & Related papers  (2023-01-14T18:58:42Z)
• "It's Not Just Hate'': A Multi-Dimensional Perspective on Detecting Harmful Speech Online [26.10949184015077]
  We show that a more fine-grained multi-label approach to predicting incivility and hateful or intolerant content addresses both conceptual and performance issues. We release a novel dataset of over 40,000 tweets about immigration from the US and UK, annotated with six labels for different aspects of incivility and intolerance.
  arXiv  Detail & Related papers  (2022-10-28T03:34:50Z)
• Low Government Performance and Uncivil Political Posts on Social Media: Evidence from the COVID-19 Crisis in the US [0.0]
  It is less clear how a government's performance is linked with people's uncivil political expression on social media. The present study collected over 8 million posts on X/Twitter directed at US state governors and classified them as uncivil or not. The results of the statistical analyses showed that increases in state-level COVID-19 cases led to a significantly higher number of uncivil posts against state governors.
  arXiv  Detail & Related papers  (2021-07-21T12:19:14Z)
• Towards Understanding and Mitigating Social Biases in Language Models [107.82654101403264]
  Large-scale pretrained language models (LMs) can be potentially dangerous in manifesting undesirable representational biases. We propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information.
  arXiv  Detail & Related papers  (2021-06-24T17:52:43Z)
• Causal Understanding of Fake News Dissemination on Social Media [50.4854427067898]
  We argue that it is critical to understand what user attributes potentially cause users to share fake news. In fake news dissemination, confounders can be characterized by fake news sharing behavior that inherently relates to user attributes and online activities. We propose a principled approach to alleviating selection bias in fake news dissemination.
  arXiv  Detail & Related papers  (2020-10-20T19:37:04Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)
• Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News [67.53424807783414]
  Social media has greatly enabled people to participate in online activities at an unprecedented rate. This unrestricted access also exacerbates the spread of misinformation and fake news online which might cause confusion and chaos unless being detected early for its mitigation. We jointly leverage the limited amount of clean data along with weak signals from social engagements to train deep neural networks in a meta-learning framework to estimate the quality of different weak instances. Experiments on realworld datasets demonstrate that the proposed framework outperforms state-of-the-art baselines for early detection of fake news without using any user engagements at prediction time.
  arXiv  Detail & Related papers  (2020-04-03T18:26:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.