Are We Falling in a Middle-Intelligence Trap? An Analysis and Mitigation of the Reversal Curse

• URL: http://arxiv.org/abs/2311.07468v2
• Date: Thu, 16 Nov 2023 08:35:05 GMT
• Title: Are We Falling in a Middle-Intelligence Trap? An Analysis and Mitigation of the Reversal Curse
• Authors: Ang Lv and Kaiyi Zhang and Shufang Xie and Quan Tu and Yuhan Chen and Ji-Rong Wen and Rui Yan
• Abstract summary: Recent studies have highlighted a phenomenon in large language models known as "the reversal curse" We contend that the reversal curse is partially a result of specific model training objectives. We propose a novel training method, BI Casual language modeling Optimization (BICO), designed to mitigate the reversal curse.
• Score: 73.65112477688353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies have highlighted a phenomenon in large language models (LLMs) known as "the reversal curse," in which the order of knowledge entities in the training data biases the models' comprehension. For example, if a model is trained on sentences where entity A consistently appears before entity B, it can respond to queries about A by providing B as the answer. However, it may encounter confusion when presented with questions concerning B. We contend that the reversal curse is partially a result of specific model training objectives, particularly evident in the prevalent use of the next-token prediction within most causal language models. For the next-token prediction, models solely focus on a token's preceding context, resulting in a restricted comprehension of the input. In contrast, we illustrate that the GLM, trained using the autoregressive blank infilling objective where tokens to be predicted have access to the entire context, exhibits better resilience against the reversal curse. We propose a novel training method, BIdirectional Casual language modeling Optimization (BICO), designed to mitigate the reversal curse when fine-tuning pretrained causal language models on new data. BICO modifies the causal attention mechanism to function bidirectionally and employs a mask denoising optimization. In the task designed to assess the reversal curse, our approach improves Llama's accuracy from the original 0% to around 70%. We hope that more attention can be focused on exploring and addressing these inherent weaknesses of the current LLMs, in order to achieve a higher level of intelligence.

Related papers

• The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More [27.731438642876114]
  We study the reversal curse, where models cannot recall information when probed in a different order than was encountered during training. We find that the factorization curse is an inherent failure of the next-token prediction objective used in popular large language models. Our results uncover a promising path forward: factorization-agnostic objectives can significantly mitigate the reversal curse and hint at improved knowledge storage and planning capabilities.
  arXiv  Detail & Related papers  (2024-06-07T18:00:37Z)
• Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training [57.771940716189114]
  We show that large language models (LLMs) suffer from the "reversal curse" The root cause of the reversal curse lies in the different word order between the training and inference stage. We propose Semantic-aware Permutation Training (SPT) to address this issue.
  arXiv  Detail & Related papers  (2024-03-01T18:55:20Z)
• Fine-tuning Language Models for Factuality [96.5203774943198]
  Large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations' In this work, we fine-tune language models to be more factual, without human labeling.
  arXiv  Detail & Related papers  (2023-11-14T18:59:15Z)
• Anti-LM Decoding for Zero-shot In-context Machine Translation [59.26037416204157]
  This work introduces an Anti-Language Model objective with a decay factor designed to address the weaknesses of In-context Machine Translation. We conduct experiments across 3 model types and sizes, 3 language directions, and for both greedy decoding and beam search.
  arXiv  Detail & Related papers  (2023-11-14T17:09:43Z)
• Improving Input-label Mapping with Demonstration Replay for In-context Learning [67.57288926736923]
  In-context learning (ICL) is an emerging capability of large autoregressive language models. We propose a novel ICL method called Sliding Causal Attention (RdSca) We show that our method significantly improves the input-label mapping in ICL demonstrations.
  arXiv  Detail & Related papers  (2023-10-30T14:29:41Z)
• MiLe Loss: a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models [40.992566245706996]
  We propose a MiLe Loss function for mitigating the bias of learning difficulties with tokens. We train generative language models at different scales of 468M, 1.2B, and 6.7B parameters. Experiments reveal that models incorporating the proposed MiLe Loss can gain consistent performance improvement on downstream benchmarks.
  arXiv  Detail & Related papers  (2023-10-30T13:33:21Z)
• GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator [114.8954615026781]
  We propose a GAN-style model for encoder-decoder pre-training by introducing an auxiliary discriminator. GanLM is trained with two pre-training objectives: replaced token detection and replaced token denoising. Experiments in language generation benchmarks show that GanLM with the powerful language understanding capability outperforms various strong pre-trained language models.
  arXiv  Detail & Related papers  (2022-12-20T12:51:11Z)
• Cold-start Active Learning through Self-supervised Language Modeling [15.551710499866239]
  Active learning aims to reduce annotation costs by choosing the most critical examples to label. With BERT, we develop a simple strategy based on the masked language modeling loss. Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and time.
  arXiv  Detail & Related papers  (2020-10-19T14:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.