Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models

• URL: http://arxiv.org/abs/2405.05417v2
• Date: Fri, 27 Sep 2024 09:03:05 GMT
• Title: Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
• Authors: Sander Land, Max Bartolo,
• Abstract summary: The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model behaviour. We present a comprehensive analysis of Large Language Model tokenizers, specifically targeting this issue of detecting under-trained tokens. Through a combination of tokenizer analysis, model weight-based indicators, and prompting techniques, we develop novel and effective methods for automatically detecting these problematic tokens.
• Score: 4.165536532090932
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model behaviour. Although such `glitch tokens', tokens present in the tokenizer vocabulary but that are nearly or entirely absent during model training, have been observed across various models, a reliable method to identify and address them has been missing. We present a comprehensive analysis of Large Language Model tokenizers, specifically targeting this issue of detecting under-trained tokens. Through a combination of tokenizer analysis, model weight-based indicators, and prompting techniques, we develop novel and effective methods for automatically detecting these problematic tokens. Our findings demonstrate the prevalence of such tokens across a diverse set of models and provide insights into improving the efficiency and safety of language models.

Related papers

• FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers [55.2480439325792]
  We propose FUSE, an approach to approximating an adapter layer that maps from one model's textual embedding space to another, even across different tokenizers. We show the efficacy of our approach via multi-objective optimization over vision-language and causal language models for image captioning and sentiment-based image captioning.
  arXiv  Detail & Related papers  (2024-08-09T02:16:37Z)
• Understanding and Mitigating Tokenization Bias in Language Models [6.418593476658017]
  State-of-the-art language models are autoregressive and operate on subword units known as tokens. We show that popular encoding schemes induce a sampling bias that cannot be mitigated with more training or data. We propose a novel algorithm to obtain unbiased estimates from any language model trained on tokenized data.
  arXiv  Detail & Related papers  (2024-06-24T17:38:02Z)
• Few-Shot Detection of Machine-Generated Text using Style Representations [4.326503887981912]
  Language models that convincingly mimic human writing pose a significant risk of abuse. We propose to leverage representations of writing style estimated from human-authored text. We find that features effective at distinguishing among human authors are also effective at distinguishing human from machine authors.
  arXiv  Detail & Related papers  (2024-01-12T17:26:51Z)
• Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation [82.5217996570387]
  We adapt a pre-trained language model for auto-regressive text-to-image generation. We find that pre-trained language models offer limited help.
  arXiv  Detail & Related papers  (2023-11-27T07:19:26Z)
• Improving Input-label Mapping with Demonstration Replay for In-context Learning [67.57288926736923]
  In-context learning (ICL) is an emerging capability of large autoregressive language models. We propose a novel ICL method called Sliding Causal Attention (RdSca) We show that our method significantly improves the input-label mapping in ICL demonstrations.
  arXiv  Detail & Related papers  (2023-10-30T14:29:41Z)
• Quark: Controllable Text Generation with Reinforced Unlearning [68.07749519374089]
  Large-scale language models often learn behaviors that are misaligned with user expectations. We introduce Quantized Reward Konditioning (Quark), an algorithm for optimizing a reward function that quantifies an (un)wanted property. For unlearning toxicity, negative sentiment, and repetition, our experiments show that Quark outperforms both strong baselines and state-of-the-art reinforcement learning methods.
  arXiv  Detail & Related papers  (2022-05-26T21:11:51Z)
• Pre-trained Token-replaced Detection Model as Few-shot Learner [31.40447168356879]
  We propose a novel approach to few-shot learning with pre-trained token-replaced detection models like ELECTRA. A systematic evaluation on 16 datasets demonstrates that our approach outperforms few-shot learners with pre-trained masked language models.
  arXiv  Detail & Related papers  (2022-03-07T09:47:53Z)
• Interpreting Language Models with Contrastive Explanations [99.7035899290924]
  Language models must consider various features to predict a token, such as its part of speech, number, tense, or semantics. Existing explanation methods conflate evidence for all these features into a single explanation, which is less interpretable for human understanding. We show that contrastive explanations are quantifiably better than non-contrastive explanations in verifying major grammatical phenomena.
  arXiv  Detail & Related papers  (2022-02-21T18:32:24Z)
• Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models [19.21465581259624]
  Many problems can be attributed to models exploiting spurious correlations, or shortcuts between the training data and the task labels. In this paper, we aim to automatically identify such spurious correlations in NLP models at scale. We show that our proposed method can effectively and efficiently identify a scalable set of "shortcuts", and mitigating these leads to more robust models in multiple applications.
  arXiv  Detail & Related papers  (2021-10-14T21:40:03Z)
• AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding [0.0]
  We develop an Arabic language representation model, which we name AraELECTRA. Our model is pretrained using the replaced token detection objective on large Arabic text corpora. We show that AraELECTRA outperforms current state-of-the-art Arabic language representation models, given the same pretraining data and with even a smaller model size.
  arXiv  Detail & Related papers  (2020-12-31T09:35:39Z)
• Word Shape Matters: Robust Machine Translation with Visual Embedding [78.96234298075389]
  We introduce a new encoding of the input symbols for character-level NLP models. It encodes the shape of each character through the images depicting the letters when printed. We name this new strategy visual embedding and it is expected to improve the robustness of NLP models.
  arXiv  Detail & Related papers  (2020-10-20T04:08:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.