TokenBreak: Bypassing Text Classification Models Through Token Manipulation

• URL: http://arxiv.org/abs/2506.07948v1
• Date: Mon, 09 Jun 2025 17:11:28 GMT
• Title: TokenBreak: Bypassing Text Classification Models Through Token Manipulation
• Authors: Kasimir Schulz, Kenneth Yeung, Kieran Evans,
• Abstract summary: Text classification models can be implemented to guard against threats such as prompt injection attacks against Large Language Models (LLMs)<n>We introduce TokenBreak: a novel attack that can bypass these protection models by taking advantage of the tokenization strategy they use.<n> Importantly, the end target (LLM or email recipient) can still understand and respond to the manipulated text and therefore be vulnerable to the very attack the protection model was put in place to prevent.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Natural Language Processing (NLP) models are used for text-related tasks such as classification and generation. To complete these tasks, input data is first tokenized from human-readable text into a format the model can understand, enabling it to make inferences and understand context. Text classification models can be implemented to guard against threats such as prompt injection attacks against Large Language Models (LLMs), toxic input and cybersecurity risks such as spam emails. In this paper, we introduce TokenBreak: a novel attack that can bypass these protection models by taking advantage of the tokenization strategy they use. This attack technique manipulates input text in such a way that certain models give an incorrect classification. Importantly, the end target (LLM or email recipient) can still understand and respond to the manipulated text and therefore be vulnerable to the very attack the protection model was put in place to prevent. The tokenizer is tied to model architecture, meaning it is possible to predict whether or not a model is vulnerable to attack based on family. We also present a defensive strategy as an added layer of protection that can be implemented without having to retrain the defensive model.

Related papers

• No Query, No Access [50.18709429731724]
  We introduce the textbfVictim Data-based Adrial Attack (VDBA), which operates using only victim texts.<n>To prevent access to the victim model, we create a shadow dataset with publicly available pre-trained models and clustering methods.<n>Experiments on the Emotion and SST5 datasets show that VDBA outperforms state-of-the-art methods, achieving an ASR improvement of 52.08%.
  arXiv  Detail & Related papers  (2025-05-12T06:19:59Z)
• An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks [87.64278063236847]
  In this work, we propose a unified threat model for the principled comparison of jailbreak attacks.<n>Our threat model checks if a given jailbreak is likely to occur in the distribution of text.<n>We adapt popular attacks to this threat model, and, for the first time, benchmark these attacks on equal footing with it.
  arXiv  Detail & Related papers  (2024-10-21T17:27:01Z)
• Defense Against Syntactic Textual Backdoor Attacks with Token Substitution [15.496176148454849]
  It embeds carefully chosen triggers into a victim model at the training stage, and makes the model erroneously predict inputs containing the same triggers as a certain class. This paper proposes a novel online defense algorithm that effectively counters syntax-based as well as special token-based backdoor attacks.
  arXiv  Detail & Related papers  (2024-07-04T22:48:57Z)
• Query-Based Adversarial Prompt Generation [72.06860443442429]
  We build adversarial examples that cause an aligned language model to emit harmful strings.<n>We validate our attack on GPT-3.5 and OpenAI's safety classifier.
  arXiv  Detail & Related papers  (2024-02-19T18:01:36Z)
• OrderBkd: Textual backdoor attack through repositioning [0.0]
  Third-party datasets and pre-trained machine learning models pose a threat to NLP systems. Existing backdoor attacks involve poisoning the data samples such as insertion of tokens or sentence paraphrasing. Our main difference from the previous work is that we use the reposition of a two words in a sentence as a trigger.
  arXiv  Detail & Related papers  (2024-02-12T14:53:37Z)
• Are aligned neural networks adversarially aligned? [93.91072860401856]
  adversarial users can construct inputs which circumvent attempts at alignment. We show that existing NLP-based optimization attacks are insufficiently powerful to reliably attack aligned text models. We conjecture that improved NLP attacks may demonstrate this same level of adversarial control over text-only models.
  arXiv  Detail & Related papers  (2023-06-26T17:18:44Z)
• Defense-Prefix for Preventing Typographic Attacks on CLIP [14.832208701208414]
  Some adversarial attacks fool a model into false or absurd classifications. We introduce our simple yet effective method: Defense-Prefix (DP), which inserts the DP token before a class name to make words "robust" against typographic attacks. Our method significantly improves the accuracy of classification tasks for typographic attack datasets, while maintaining the zero-shot capabilities of the model.
  arXiv  Detail & Related papers  (2023-04-10T11:05:20Z)
• PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models [9.630961791758168]
  Malicious users can evade deep detection models by manipulating their behavior. Here we create a novel adversarial attack model against deep user sequence embedding-based classification models. In the attack, the adversary generates a new post to fool the classifier.
  arXiv  Detail & Related papers  (2021-09-14T15:48:07Z)
• Towards Variable-Length Textual Adversarial Attacks [68.27995111870712]
  It is non-trivial to conduct textual adversarial attacks on natural language processing tasks due to the discreteness of data. In this paper, we propose variable-length textual adversarial attacks(VL-Attack) Our method can achieve $33.18$ BLEU score on IWSLT14 German-English translation, achieving an improvement of $1.47$ over the baseline model.
  arXiv  Detail & Related papers  (2021-04-16T14:37:27Z)
• Hidden Backdoor Attack against Semantic Segmentation Models [60.0327238844584]
  The emphbackdoor attack intends to embed hidden backdoors in deep neural networks (DNNs) by poisoning training data. We propose a novel attack paradigm, the emphfine-grained attack, where we treat the target label from the object-level instead of the image-level. Experiments show that the proposed methods can successfully attack semantic segmentation models by poisoning only a small proportion of training data.
  arXiv  Detail & Related papers  (2021-03-06T05:50:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.