Towards Faster k-Nearest-Neighbor Machine Translation

• URL: http://arxiv.org/abs/2312.07419v1
• Date: Tue, 12 Dec 2023 16:41:29 GMT
• Title: Towards Faster k-Nearest-Neighbor Machine Translation
• Authors: Xiangyu Shi, Yunlong Liang, Jinan Xu, Yufeng Chen
• Abstract summary: k-nearest-neighbor machine translation approaches suffer from heavy retrieve overhead on the entire datastore when decoding each token. We propose a simple yet effective multi-layer perceptron (MLP) network to predict whether a token should be translated jointly by the neural machine translation model and probabilities produced by the kNN.
• Score: 56.66038663128903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent works have proven the effectiveness of k-nearest-neighbor machine translation(a.k.a kNN-MT) approaches to produce remarkable improvement in cross-domain translations. However, these models suffer from heavy retrieve overhead on the entire datastore when decoding each token. We observe that during the decoding phase, about 67% to 84% of tokens are unvaried after searching over the corpus datastore, which means most of the tokens cause futile retrievals and introduce unnecessary computational costs by initiating k-nearest-neighbor searches. We consider this phenomenon is explainable in linguistics and propose a simple yet effective multi-layer perceptron (MLP) network to predict whether a token should be translated jointly by the neural machine translation model and probabilities produced by the kNN or just by the neural model. The results show that our method succeeds in reducing redundant retrieval operations and significantly reduces the overhead of kNN retrievals by up to 53% at the expense of a slight decline in translation quality. Moreover, our method could work together with all existing kNN-MT systems.

Related papers

• Simply Trainable Nearest Neighbour Machine Translation with GPU Inference [2.3420045370973828]
  This paper proposes a trainable nearest neighbor machine translation on GPU. We first adaptively construct a small datastore for each input sentence. Second, we train a single-layer network for the adaption between the knnMT and pre-trained result to automatically interpolate in different domains.
  arXiv  Detail & Related papers  (2024-07-29T12:55:40Z)
• knn-seq: Efficient, Extensible kNN-MT Framework [11.421689052786467]
  k-nearest-neighbor machine translation (kNN-MT) boosts the translation quality of a pre-trained machine translation (NMT) model by utilizing translation examples during decoding. Due to its size, it is computationally expensive both to construct and to retrieve examples from the datastore. We present an efficient and achieves kNN-MT framework, knn-seq, for researchers and developers that is carefully designed to run efficiently, even with a billion-scale large datastore.
  arXiv  Detail & Related papers  (2023-10-18T21:56:04Z)
• Nearest Neighbor Zero-Shot Inference [68.56747574377215]
  kNN-Prompt is a technique to use k-nearest neighbor (kNN) retrieval augmentation for zero-shot inference with language models (LMs) fuzzy verbalizers leverage the sparse kNN distribution for downstream tasks by automatically associating each classification label with a set of natural language tokens. Experiments show that kNN-Prompt is effective for domain adaptation with no further training, and that the benefits of retrieval increase with the size of the model used for kNN retrieval.
  arXiv  Detail & Related papers  (2022-05-27T07:00:59Z)
• Efficient Cluster-Based k-Nearest-Neighbor Machine Translation [65.69742565855395]
  k-Nearest-Neighbor Machine Translation (kNN-MT) has been recently proposed as a non-parametric solution for domain adaptation in neural machine translation (NMT)
  arXiv  Detail & Related papers  (2022-04-13T05:46:31Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Adaptive Nearest Neighbor Machine Translation [60.97183408140499]
  kNN-MT combines pre-trained neural machine translation with token-level k-nearest-neighbor retrieval. Traditional kNN algorithm simply retrieves a same number of nearest neighbors for each target token. We propose Adaptive kNN-MT to dynamically determine the number of k for each target token.
  arXiv  Detail & Related papers  (2021-05-27T09:27:42Z)
• Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval [45.88734029123836]
  We propose a novel approach to alleviate problems by limiting the open target vocabulary search space of QT to a set of important words mined from search index database. The proposed methods are exploited and examined in a real-word CLIR system--Aliexpress e-Commerce search engine.
  arXiv  Detail & Related papers  (2020-10-26T15:27:51Z)
• Nearest Neighbor Machine Translation [113.96357168879548]
  We introduce $k$-nearest-neighbor machine translation ($k$NN-MT) It predicts tokens with a nearest neighbor classifier over a large datastore of cached examples. It consistently improves performance across many settings.
  arXiv  Detail & Related papers  (2020-10-01T22:24:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.