Mitigating Copy Bias in In-Context Learning through Neuron Pruning

• URL: http://arxiv.org/abs/2410.01288v2
• Date: Thu, 3 Oct 2024 07:13:43 GMT
• Title: Mitigating Copy Bias in In-Context Learning through Neuron Pruning
• Authors: Ameen Ali, Lior Wolf, Ivan Titov,
• Abstract summary: Large language models (LLMs) have demonstrated impressive few-shot in-context learning abilities. They are sometimes prone to a copying bias', where they copy answers from provided examples instead of learning the underlying patterns. We propose a novel and simple method to mitigate such copying bias.
• Score: 74.91243772654519
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated impressive few-shot in-context learning (ICL) abilities. Still, we show that they are sometimes prone to a `copying bias', where they copy answers from provided examples instead of learning the underlying patterns. In this work, we propose a novel and simple method to mitigate such copying bias. First, we create a synthetic task and use the Integrated Gradients method to identify neurons that prioritize copying over generalization. We demonstrate that pruning these neurons consistently improves performance across a diverse set of ICL tasks. We also show that our method is applicable across various LLM architectures, including Transformers and State-Space Models, without requiring modifications. In our analysis, we adopt a task-recognition perspective on ICL and examine task vectors (Hendel et al., 2023) induced by the model. We find that pruning enhances the quality of these vectors, suggesting that the pruned neurons previously hindered effective task recognition.

Related papers

• Debiasing Multimodal Large Language Models [61.6896704217147]
  Large Vision-Language Models (LVLMs) have become indispensable tools in computer vision and natural language processing. Our investigation reveals a noteworthy bias in the generated content, where the output is primarily influenced by the underlying Large Language Models (LLMs) prior to the input image. To rectify these biases and redirect the model's focus toward vision information, we introduce two simple, training-free strategies.
  arXiv  Detail & Related papers  (2024-03-08T12:35:07Z)
• Class incremental learning with probability dampening and cascaded gated classifier [4.285597067389559]
  We propose a novel incremental regularisation approach called Margin Dampening and Cascaded Scaling. The first combines a soft constraint and a knowledge distillation approach to preserve past knowledge while allowing forgetting new patterns. We empirically show that our approach performs well on multiple benchmarks well-established baselines.
  arXiv  Detail & Related papers  (2024-02-02T09:33:07Z)
• Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study [68.75670223005716]
  We find that pre-trained language models like BERT have a potential ability to learn sequentially, even without any sparse memory replay. Our experiments reveal that BERT can actually generate high quality representations for previously learned tasks in a long term, under extremely sparse replay or even no replay.
  arXiv  Detail & Related papers  (2023-03-02T09:03:43Z)
• Measures of Information Reflect Memorization Patterns [53.71420125627608]
  We show that the diversity in the activation patterns of different neurons is reflective of model generalization and memorization. Importantly, we discover that information organization points to the two forms of memorization, even for neural activations computed on unlabelled in-distribution examples.
  arXiv  Detail & Related papers  (2022-10-17T20:15:24Z)
• Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports [2.5953185061765884]
  We show that a single multi-tasking model can match the performance of task specific models. We validate our observations on our internal radiologist-annotated datasets on the cervical and lumbar spine.
  arXiv  Detail & Related papers  (2022-05-06T01:51:19Z)
• Towards Sample-efficient Overparameterized Meta-learning [37.676063120293044]
  An overarching goal in machine learning is to build a generalizable model with few samples. This paper aims to demystify over parameterization for meta-learning. We show that learning the optimal representation coincides with the problem of designing a task-aware regularization.
  arXiv  Detail & Related papers  (2022-01-16T21:57:17Z)
• Learning Invariant Representation for Continual Learning [5.979373021392084]
  A key challenge in Continual learning is catastrophically forgetting previously learned tasks when the agent faces a new one. We propose a new pseudo-rehearsal-based method, named learning Invariant Representation for Continual Learning (IRCL) Disentangling the shared invariant representation helps to learn continually a sequence of tasks, while being more robust to forgetting and having better knowledge transfer.
  arXiv  Detail & Related papers  (2021-01-15T15:12:51Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.