Repetition Neurons: How Do Language Models Produce Repetitions?

• URL: http://arxiv.org/abs/2410.13497v1
• Date: Thu, 17 Oct 2024 12:43:47 GMT
• Title: Repetition Neurons: How Do Language Models Produce Repetitions?
• Authors: Tatsuya Hiraoka, Kentaro Inui,
• Abstract summary: This paper introduces repetition neurons, regarded as skill neurons responsible for the repetition problem in text generation tasks. We identify these repetition neurons by comparing activation values before and after the onset of repetition in texts generated by recent pre-trained language models.
• Score: 25.430820735194768
• License:
• Abstract: This paper introduces repetition neurons, regarded as skill neurons responsible for the repetition problem in text generation tasks. These neurons are progressively activated more strongly as repetition continues, indicating that they perceive repetition as a task to copy the previous context repeatedly, similar to in-context learning. We identify these repetition neurons by comparing activation values before and after the onset of repetition in texts generated by recent pre-trained language models. We analyze the repetition neurons in three English and one Japanese pre-trained language models and observe similar patterns across them.

Related papers

• From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty [67.81977289444677]
  Large language models (LLMs) often exhibit undesirable behaviors, such as hallucinations and sequence repetitions. We categorize fallback behaviors -- sequence repetitions, degenerate text, and hallucinations -- and extensively analyze them. Our experiments reveal a clear and consistent ordering of fallback behaviors, across all these axes.
  arXiv  Detail & Related papers  (2024-07-08T16:13:42Z)
• Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective [91.14291142262262]
  This work presents a straightforward and fundamental explanation from the data perspective. Our preliminary investigation reveals a strong correlation between the degeneration issue and the presence of repetitions in training data. Our experiments reveal that penalizing the repetitions in training data remains critical even when considering larger model sizes and instruction tuning.
  arXiv  Detail & Related papers  (2023-10-16T09:35:42Z)
• Measures of Information Reflect Memorization Patterns [53.71420125627608]
  We show that the diversity in the activation patterns of different neurons is reflective of model generalization and memorization. Importantly, we discover that information organization points to the two forms of memorization, even for neural activations computed on unlabelled in-distribution examples.
  arXiv  Detail & Related papers  (2022-10-17T20:15:24Z)
• Generating Repetitions with Appropriate Repeated Words [30.10429353715689]
  Repetitions are essential in communication to build trust with others. To the best of our knowledge, this is the first neural approach to address repetition generation. We propose Weighted Label Smoothing, a smoothing method for explicitly learning which words to repeat during fine-tuning, and a repetition scoring method that can output more appropriate repetitions during decoding.
  arXiv  Detail & Related papers  (2022-07-03T01:21:49Z)
• Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation [41.3948101212288]
  We study the relationship between the probabilities of the repetitive tokens and their previous repetitions in the context. We propose a training method where the model learns to penalize probabilities of sentence-level repetitions from pseudo repetitive data.
  arXiv  Detail & Related papers  (2022-06-06T05:51:12Z)
• Counterfactual Memorization in Neural Language Models [91.8747020391287]
  Modern neural language models that are widely used in various NLP tasks risk memorizing sensitive information from their training data. An open question in previous studies of language model memorization is how to filter out "common" memorization. We formulate a notion of counterfactual memorization which characterizes how a model's predictions change if a particular document is omitted during training.
  arXiv  Detail & Related papers  (2021-12-24T04:20:57Z)
• Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects [82.81964713263483]
  A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli.
  arXiv  Detail & Related papers  (2021-10-12T15:30:21Z)
• On-the-Fly Attention Modularization for Neural Generation [54.912042110885366]
  We show that generated text is repetitive, generic, self-inconsistent, and lacking commonsense. Our findings motivate on-the-fly attention modularization, a simple but effective method for injecting inductive biases into attention during inference.
  arXiv  Detail & Related papers  (2021-01-02T05:16:46Z)
• Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration [45.14559188965439]
  We propose to apply reinforcement learning to refine an MLE-based language model without user simulators. We distill sentence-level information about repetition, inconsistency and task relevance through rewards. Experiments show that our model outperforms previous state-of-the-art dialogue models on both automatic metrics and human evaluation results.
  arXiv  Detail & Related papers  (2020-12-31T00:02:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.