Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else?

• URL: http://arxiv.org/abs/2111.05139v1
• Date: Tue, 9 Nov 2021 13:30:34 GMT
• Title: Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else?
• Authors: Alexander Michael Daniel
• Abstract summary: Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system.
• Score: 93.91375268580806
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, however, the canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system. This paper aims to determine which of these techniques is best suited for this purpose and how each technique might best be used towards this end. Training datasets of the same size and nearly identical neural architectures (a BERT transformer as a word embedder with a single feed-forward layer thereafter) are used for each approach, which are then tested on sentiment- and stance-specific datasets to establish a baseline of how well each method can be used to do the other tasks. Four different datasets relating to COVID-19 disinformation are used to test the ability of each technique to detect disinformation on a topic that did not appear in the training data set. Quantitative and qualitative results from these tests are then used to provide insight into how best to employ these techniques in practice.

Related papers

• Approaching Metaheuristic Deep Learning Combos for Automated Data Mining [0.5419570023862531]
  This work proposes a means of combining meta-heuristic methods with conventional classifiers and neural networks in order to perform automated data mining. Experiments on the MNIST dataset for handwritten digit recognition were performed. It was empirically observed that using a ground truth labeled dataset's validation accuracy is inadequate for correcting labels of other previously unseen data instances.
  arXiv  Detail & Related papers  (2024-10-16T10:28:22Z)
• Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future [119.88454942558485]
  Underwater object detection (UOD) aims to identify and localise objects in underwater images or videos. In recent years, artificial intelligence (AI) based methods, especially deep learning methods, have shown promising performance in UOD.
  arXiv  Detail & Related papers  (2024-10-08T00:25:33Z)
• Self-supervised Pre-training of Text Recognizers [0.0]
  We study self-supervised pre-training methods based on masked label prediction. We perform experiments on historical handwritten (Bentham) and historical printed datasets. The evaluation shows that the self-supervised pre-training on data from the target domain is very effective, but it struggles to outperform transfer learning from closely related domains.
  arXiv  Detail & Related papers  (2024-05-01T09:58:57Z)
• Multi-Dimensional Ability Diagnosis for Machine Learning Algorithms [88.93372675846123]
  We propose a task-agnostic evaluation framework Camilla for evaluating machine learning algorithms. We use cognitive diagnosis assumptions and neural networks to learn the complex interactions among algorithms, samples and the skills of each sample. In our experiments, Camilla outperforms state-of-the-art baselines on the metric reliability, rank consistency and rank stability.
  arXiv  Detail & Related papers  (2023-07-14T03:15:56Z)
• Algorithmic failure as a humanities methodology: machine learning's mispredictions identify rich cases for qualitative analysis [0.0]
  I trained a simple machine learning algorithm to predict whether or not an action was active or passive using only information about fictional characters. The results thus support Munk et al.'s theory that failed predictions can be productively used to identify rich cases for qualitative analysis. Further research is needed to develop an understanding of what kinds of data the method is useful for and which kinds of machine learning are most generative.
  arXiv  Detail & Related papers  (2023-05-19T13:24:32Z)
• Panning for gold: Lessons learned from the platform-agnostic automated detection of political content in textual data [48.7576911714538]
  We discuss how these techniques can be used to detect political content across different platforms. We compare the performance of three groups of detection techniques relying on dictionaries, supervised machine learning, or neural networks. Our results show the limited impact of preprocessing on model performance, with the best results for less noisy data being achieved by neural network- and machine-learning-based models.
  arXiv  Detail & Related papers  (2022-07-01T15:23:23Z)
• Finding Facial Forgery Artifacts with Parts-Based Detectors [73.08584805913813]
  We design a series of forgery detection systems that each focus on one individual part of the face. We use these detectors to perform detailed empirical analysis on the FaceForensics++, Celeb-DF, and Facebook Deepfake Detection Challenge datasets.
  arXiv  Detail & Related papers  (2021-09-21T16:18:45Z)
• Technological Approaches to Detecting Online Disinformation and Manipulation [0.0]
  The move of propaganda and disinformation to the online environment is possible thanks to the fact that within the last decade, digital information channels radically increased in popularity as a news source. In this chapter, an overview of computer-supported approaches to detecting disinformation and manipulative techniques based on several criteria is presented.
  arXiv  Detail & Related papers  (2021-08-26T09:28:50Z)
• Combining Feature and Instance Attribution to Detect Artifacts [62.63504976810927]
  We propose methods to facilitate identification of training data artifacts. We show that this proposed training-feature attribution approach can be used to uncover artifacts in training data. We execute a small user study to evaluate whether these methods are useful to NLP researchers in practice.
  arXiv  Detail & Related papers  (2021-07-01T09:26:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.