Meta-Learning Online Adaptation of Language Models
- URL: http://arxiv.org/abs/2305.15076v2
- Date: Fri, 20 Oct 2023 22:49:24 GMT
- Title: Meta-Learning Online Adaptation of Language Models
- Authors: Nathan Hu, Eric Mitchell, Christopher D. Manning, Chelsea Finn
- Abstract summary: Large language models encode impressively broad world knowledge in their parameters.
However, the knowledge in static language models falls out of date, limiting the model's effective "shelf life"
- Score: 88.8947656843812
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models encode impressively broad world knowledge in their
parameters. However, the knowledge in static language models falls out of date,
limiting the model's effective "shelf life." While online fine-tuning can
reduce this degradation, we find that naively fine-tuning on a stream of
documents leads to a low level of information uptake. We hypothesize that
online fine-tuning does not sufficiently attend to important information. That
is, the gradient signal from important tokens representing factual information
is drowned out by the gradient from inherently noisy tokens, suggesting that a
dynamic, context-aware learning rate may be beneficial. We therefore propose
learning which tokens to upweight. We meta-train a small, autoregressive model
to reweight the language modeling loss for each token during online
fine-tuning, with the objective of maximizing the out-of-date base
question-answering model's ability to answer questions about a document after a
single weighted gradient step. We call this approach Context-aware Meta-learned
Loss Scaling (CaMeLS). Across three different distributions of documents, our
experiments find that CaMeLS provides substantially improved information uptake
on streams of thousands of documents compared with standard fine-tuning and
baseline heuristics for reweighting token losses.
Related papers
- Machine Unlearning in Large Language Models [0.7864304771129751]
This paper introduces a methodology to align large language models (LLMs) with ethical, privacy, and safety standards.
Our approach aims to selectively erase or modify learned information in LLMs, targeting harmful responses and copyrighted content.
arXiv Detail & Related papers (2024-05-24T02:12:51Z) - UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models [12.45822383965784]
We introduce UnDIAL (Unlearning via Self-Distillation on Adjusted Logits), a novel and robust unlearning method.
Our approach leverages self-distillation to adjust logits and selectively reduce the influence of targeted tokens.
arXiv Detail & Related papers (2024-02-15T16:21:14Z) - Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning
Interference with Gradient Projection [56.292071534857946]
Recent data-privacy laws have sparked interest in machine unlearning.
Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset.
We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU)
We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
arXiv Detail & Related papers (2023-12-07T07:17:24Z) - Ignorance is Bliss: Robust Control via Information Gating [60.17644038829572]
Informational parsimony provides a useful inductive bias for learning representations that achieve better generalization by being robust to noise and spurious correlations.
We propose textitinformation gating as a way to learn parsimonious representations that identify the minimal information required for a task.
arXiv Detail & Related papers (2023-03-10T18:31:50Z) - APAM: Adaptive Pre-training and Adaptive Meta Learning in Language Model
for Noisy Labels and Long-tailed Learning [9.433150673299163]
Practical natural language processing (NLP) tasks are commonly long-tailed with noisy labels.
Some commonly used resampling techniques, such as oversampling or undersampling, could easily lead to overfitting.
We propose a general framework to handle the problem of both long-tail and noisy labels.
arXiv Detail & Related papers (2023-02-06T18:40:04Z) - Meta-Learning Fast Weight Language Models [105.66999854213724]
We present Fast Weight Layers (FWLs), a neural component that provides the benefits of dynamic evaluation much more efficiently.
FWLs can be applied at training time so the model learns to make good use of gradient updates.
arXiv Detail & Related papers (2022-12-05T18:37:09Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - Machine Unlearning of Features and Labels [72.81914952849334]
We propose first scenarios for unlearning and labels in machine learning models.
Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
arXiv Detail & Related papers (2021-08-26T04:42:24Z) - Drift-Aware Multi-Memory Model for Imbalanced Data Streams [5.71097144710995]
We propose the Drift-Aware Multi-Memory Model (DAM3) to address the class imbalance problem in online learning for memory-based models.
DAM3 mitigates class imbalance by incorporating an imbalance-sensitive drift detector, preserving a balanced representation of classes in the model, and resolving retroactive interference using a working memory.
We show through experiments on real-world and synthetic datasets that the proposed method mitigates class imbalance and outperforms the state-of-the-art methods.
arXiv Detail & Related papers (2020-12-29T15:06:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.