Verifying the Robustness of Automatic Credibility Assessment

• URL: http://arxiv.org/abs/2303.08032v3
• Date: Thu, 21 Nov 2024 09:46:44 GMT
• Title: Verifying the Robustness of Automatic Credibility Assessment
• Authors: Piotr Przybyła, Alexander Shvets, Horacio Saggion,
• Abstract summary: We show that meaning-preserving changes in input text can mislead the models. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions.
• Score: 50.55687778699995
• License:
• Abstract: Text classification methods have been widely investigated as a way to detect content of low credibility: fake news, social media bots, propaganda, etc. Quite accurate models (likely based on deep neural networks) help in moderating public electronic platforms and often cause content creators to face rejection of their submissions or removal of already published texts. Having the incentive to evade further detection, content creators try to come up with a slightly modified version of the text (known as an attack with an adversarial example) that exploit the weaknesses of classifiers and result in a different output. Here we systematically test the robustness of common text classifiers against available attacking techniques and discover that, indeed, meaning-preserving changes in input text can mislead the models. The approaches we test focus on finding vulnerable spans in text and replacing individual characters or words, taking into account the similarity between the original and replacement content. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on four misinformation detection tasks in an evaluation framework designed to simulate real use-cases of content moderation. The attacked tasks include (1) fact checking and detection of (2) hyperpartisan news, (3) propaganda and (4) rumours. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions, e.g. attacks on GEMMA being up to 27\% more successful than those on BERT. Finally, we manually analyse a subset adversarial examples and check what kinds of modifications are used in successful attacks.

Related papers

• Attacking Misinformation Detection Using Adversarial Examples Generated by Language Models [0.0]
  We investigate the challenge of generating adversarial examples to test the robustness of text classification algorithms. We focus on simulation of content moderation by setting realistic limits on the number of queries an attacker is allowed to attempt.
  arXiv  Detail & Related papers  (2024-10-28T11:46:30Z)
• Forging the Forger: An Attempt to Improve Authorship Verification via Data Augmentation [52.72682366640554]
  Authorship Verification (AV) is a text classification task concerned with inferring whether a candidate text has been written by one specific author or by someone else. It has been shown that many AV systems are vulnerable to adversarial attacks, where a malicious author actively tries to fool the classifier by either concealing their writing style, or by imitating the style of another author.
  arXiv  Detail & Related papers  (2024-03-17T16:36:26Z)
• Punctuation Matters! Stealthy Backdoor Attack for Language Models [36.91297828347229]
  A backdoored model produces normal outputs on the clean samples while performing improperly on the texts. Some attack methods even cause grammatical issues or change the semantic meaning of the original texts. We propose a novel stealthy backdoor attack method against textual models, which is called textbfPuncAttack.
  arXiv  Detail & Related papers  (2023-12-26T03:26:20Z)
• Two-in-One: A Model Hijacking Attack Against Text Generation Models [19.826236952700256]
  We propose a new model hijacking attack, Ditto, that can hijack different text classification tasks into multiple generation ones. Our results show that by using Ditto, an adversary can successfully hijack text generation models without jeopardizing their utility.
  arXiv  Detail & Related papers  (2023-05-12T12:13:27Z)
• TextShield: Beyond Successfully Detecting Adversarial Sentences in Text Classification [6.781100829062443]
  Adversarial attack serves as a major challenge for neural network models in NLP, which precludes the model's deployment in safety-critical applications. Previous detection methods are incapable of giving correct predictions on adversarial sentences. We propose a saliency-based detector, which can effectively detect whether an input sentence is adversarial or not.
  arXiv  Detail & Related papers  (2023-02-03T22:58:07Z)
• Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation [33.46393193123221]
  We release a dataset for four popular attack methods on four datasets and four models. We propose a competitive baseline based on density estimation that has the highest AUC on 29 out of 30 dataset-attack-model combinations.
  arXiv  Detail & Related papers  (2022-03-03T12:32:59Z)
• Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models [86.02610674750345]
  Adversarial GLUE (AdvGLUE) is a new multi-task benchmark to explore and evaluate the vulnerabilities of modern large-scale language models under various types of adversarial attacks. We apply 14 adversarial attack methods to GLUE tasks to construct AdvGLUE, which is further validated by humans for reliable annotations. All the language models and robust training methods we tested perform poorly on AdvGLUE, with scores lagging far behind the benign accuracy.
  arXiv  Detail & Related papers  (2021-11-04T12:59:55Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Towards Variable-Length Textual Adversarial Attacks [68.27995111870712]
  It is non-trivial to conduct textual adversarial attacks on natural language processing tasks due to the discreteness of data. In this paper, we propose variable-length textual adversarial attacks(VL-Attack) Our method can achieve $33.18$ BLEU score on IWSLT14 German-English translation, achieving an improvement of $1.47$ over the baseline model.
  arXiv  Detail & Related papers  (2021-04-16T14:37:27Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.