Fast Nearest Neighbor Machine Translation

• URL: http://arxiv.org/abs/2105.14528v1
• Date: Sun, 30 May 2021 13:10:32 GMT
• Title: Fast Nearest Neighbor Machine Translation
• Authors: Yuxian Meng, Xiaoya Li, Xiayu Zheng, Fei Wu, Xiaofei Sun, Tianwei Zhang, Jiwei Li
• Abstract summary: $k$NN-MT uses the entire reference corpus as the datastore for the nearest neighbor search. Fast $k$NN-MT constructs a significantly smaller datastore for the nearest neighbor search. Fast $k$NN-MT is two-order faster than $k$NN-MT, and is only two times slower than the standard NMT model.
• Score: 30.242943649240328
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Though nearest neighbor Machine Translation ($k$NN-MT) \cite{khandelwal2020nearest} has proved to introduce significant performance boosts over standard neural MT systems, it is prohibitively slow since it uses the entire reference corpus as the datastore for the nearest neighbor search. This means each step for each beam in the beam search has to search over the entire reference corpus. $k$NN-MT is thus two-order slower than vanilla MT models, making it hard to be applied to real-world applications, especially online services. In this work, we propose Fast $k$NN-MT to address this issue. Fast $k$NN-MT constructs a significantly smaller datastore for the nearest neighbor search: for each word in a source sentence, Fast $k$NN-MT first selects its nearest token-level neighbors, which is limited to tokens that are the same as the query token. Then at each decoding step, in contrast to using the entire corpus as the datastore, the search space is limited to target tokens corresponding to the previously selected reference source tokens. This strategy avoids search through the whole datastore for nearest neighbors and drastically improves decoding efficiency. Without loss of performance, Fast $k$NN-MT is two-order faster than $k$NN-MT, and is only two times slower than the standard NMT model. Fast $k$NN-MT enables the practical use of $k$NN-MT systems in real-world MT applications.\footnote{Code is available at \url{https://github.com/ShannonAI/fast-knn-nmt.}}

Related papers

• Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval [49.825549809652436]
  $k$NN-MT constructs an external datastore to store domain-specific translation knowledge. adaptive retrieval ($k$NN-MT-AR) dynamically estimates $lambda$ and skips $k$NN retrieval if $lambda$ is less than a fixed threshold. We propose dynamic retrieval ($k$NN-MT-DR) that significantly extends vanilla $k$NN-MT in two aspects.
  arXiv  Detail & Related papers  (2024-06-10T07:36:55Z)
• Towards Faster k-Nearest-Neighbor Machine Translation [56.66038663128903]
  k-nearest-neighbor machine translation approaches suffer from heavy retrieve overhead on the entire datastore when decoding each token. We propose a simple yet effective multi-layer perceptron (MLP) network to predict whether a token should be translated jointly by the neural machine translation model and probabilities produced by the kNN.
  arXiv  Detail & Related papers  (2023-12-12T16:41:29Z)
• knn-seq: Efficient, Extensible kNN-MT Framework [11.421689052786467]
  k-nearest-neighbor machine translation (kNN-MT) boosts the translation quality of a pre-trained machine translation (NMT) model by utilizing translation examples during decoding. Due to its size, it is computationally expensive both to construct and to retrieve examples from the datastore. We present an efficient and achieves kNN-MT framework, knn-seq, for researchers and developers that is carefully designed to run efficiently, even with a billion-scale large datastore.
  arXiv  Detail & Related papers  (2023-10-18T21:56:04Z)
• INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation [57.952478914459164]
  kNN-MT has provided an effective paradigm to smooth the prediction based on neighbor representations during inference. We propose an effective training framework INK to directly smooth the representation space via adjusting representations of kNN neighbors with a small number of new parameters. Experiments on four benchmark datasets show that method achieves average gains of 1.99 COMET and 1.0 BLEU, outperforming the state-of-the-art kNN-MT system with 0.02x memory space and 1.9x inference speedup.
  arXiv  Detail & Related papers  (2023-06-10T08:39:16Z)
• Simple and Scalable Nearest Neighbor Machine Translation [11.996135740547897]
  $k$NN-MT is a powerful approach for fast domain adaptation. We propose a simple and scalable nearest neighbor machine translation framework. Our proposed approach achieves almost 90% speed as the NMT model without performance degradation.
  arXiv  Detail & Related papers  (2023-02-23T17:28:29Z)
• Better Datastore, Better Translation: Generating Datastores from Pre-Trained Models for Nearest Neural Machine Translation [48.58899349349702]
  Nearest Neighbor Machine Translation (kNNMT) is a simple and effective method of augmenting neural machine translation (NMT) with a token-level nearest neighbor retrieval mechanism. In this paper, we propose PRED, a framework that leverages Pre-trained models for Datastores in kNN-MT.
  arXiv  Detail & Related papers  (2022-12-17T08:34:20Z)
• Chunk-based Nearest Neighbor Machine Translation [7.747003493657217]
  We introduce a textitchunk-based $k$NN-MT model which retrieves chunks of tokens from the datastore, instead of a single token. Experiments on machine translation in two settings, static domain adaptation and on-the-fly'' adaptation, show that the chunk-based model leads to a significant speed-up (up to 4 times) with only a small drop in translation quality.
  arXiv  Detail & Related papers  (2022-05-24T17:39:25Z)
• Faster Nearest Neighbor Machine Translation [27.38186214015994]
  $k$NN based neural machine translation ($k$NN-MT) has achieved state-of-the-art results in a variety of MT tasks. One significant shortcoming of $k$NN-MT lies in its inefficiency in identifying the $k$ nearest neighbors of the query representation from the entire datastore. We propose textbfFaster $k$NN-MT to address this issue.
  arXiv  Detail & Related papers  (2021-12-15T14:21:26Z)
• Adaptive Nearest Neighbor Machine Translation [60.97183408140499]
  kNN-MT combines pre-trained neural machine translation with token-level k-nearest-neighbor retrieval. Traditional kNN algorithm simply retrieves a same number of nearest neighbors for each target token. We propose Adaptive kNN-MT to dynamically determine the number of k for each target token.
  arXiv  Detail & Related papers  (2021-05-27T09:27:42Z)
• Nearest Neighbor Machine Translation [113.96357168879548]
  We introduce $k$-nearest-neighbor machine translation ($k$NN-MT) It predicts tokens with a nearest neighbor classifier over a large datastore of cached examples. It consistently improves performance across many settings.
  arXiv  Detail & Related papers  (2020-10-01T22:24:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.