SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translation

• URL: http://arxiv.org/abs/2408.04872v2
• Date: Wed, 25 Sep 2024 10:00:03 GMT
• Title: SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translation
• Authors: Chenming Tang, Zhixiang Wang, Yunfang Wu,
• Abstract summary: In this work, we introduce syntactic knowledge to select better in-context examples for machine translation (MT) We propose a new strategy, namely Syntax-augmented COverage-based In-context example selection (SCOI) Our proposed SCOI obtains the highest average COMET score among all learning-free methods.
• Score: 13.87098305304058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In-context learning (ICL) greatly improves the performance of large language models (LLMs) on various down-stream tasks, where the improvement highly depends on the quality of demonstrations. In this work, we introduce syntactic knowledge to select better in-context examples for machine translation (MT). We propose a new strategy, namely Syntax-augmented COverage-based In-context example selection (SCOI), leveraging the deep syntactic structure beyond conventional word matching. Specifically, we measure the set-level syntactic coverage by computing the coverage of polynomial terms with the help of a simplified tree-to-polynomial algorithm, and lexical coverage using word overlap. Furthermore, we devise an alternate selection approach to combine both coverage measures, taking advantage of syntactic and lexical information. We conduct experiments with two multi-lingual LLMs on six translation directions. Empirical results show that our proposed SCOI obtains the highest average COMET score among all learning-free methods, indicating that combining syntactic and lexical coverage successfully helps to select better in-context examples for MT. Our code is available at https://github.com/JamyDon/SCOI.

Related papers

• Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback [50.84142264245052]
  This work introduces the Align-SLM framework to enhance the semantic understanding of textless Spoken Language Models (SLMs) Our approach generates multiple speech continuations from a given prompt and uses semantic metrics to create preference data for Direct Preference Optimization (DPO) We evaluate the framework using ZeroSpeech 2021 benchmarks for lexical and syntactic modeling, the spoken version of the StoryCloze dataset for semantic coherence, and other speech generation metrics, including the GPT4-o score and human evaluation.
  arXiv  Detail & Related papers  (2024-11-04T06:07:53Z)
• In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation [20.704153242284114]
  We focus on machine translation (MT), a task that has been shown to benefit from in-context translation examples. No systematic studies have been published on how best to select examples, and mixed results have been reported on the usefulness of similarity-based selection. We find that sentence embedding similarity can improve MT, especially for low-resource language directions.
  arXiv  Detail & Related papers  (2024-08-01T09:07:32Z)
• DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning [19.93800175353809]
  DeTriever is a novel demonstration retrieval framework that learns a weighted combination of hidden states. Our method significantly outperforms the state-of-the-art baselines on one-shot NL2 tasks.
  arXiv  Detail & Related papers  (2024-06-12T06:33:54Z)
• Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learning [38.89119606657543]
  In contrast to sentence-level translation, document-level translation (DOCMT) by large language models (LLMs) based on in-context learning faces two major challenges. We propose a Context-Aware Prompting method (CAP) to generate more accurate, cohesive, and coherent translations via in-context learning. We conduct extensive experiments across various DOCMT tasks, and the results demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-06-11T09:11:17Z)
• Text-Video Retrieval with Global-Local Semantic Consistent Learning [122.15339128463715]
  We propose a simple yet effective method, Global-Local Semantic Consistent Learning (GLSCL) GLSCL capitalizes on latent shared semantics across modalities for text-video retrieval. Our method achieves comparable performance with SOTA as well as being nearly 220 times faster in terms of computational cost.
  arXiv  Detail & Related papers  (2024-05-21T11:59:36Z)
• ParaICL: Towards Robust Parallel In-Context Learning [74.38022919598443]
  Large language models (LLMs) have become the norm in natural language processing. Few-shot in-context learning (ICL) relies on the choice of few-shot demonstration examples. We propose a novel method named parallel in-context learning (ParaICL)
  arXiv  Detail & Related papers  (2024-03-31T05:56:15Z)
• Going Beyond Word Matching: Syntax Improves In-context Example Selection for Machine Translation [13.87098305304058]
  In-context learning (ICL) is the trending prompting strategy in the era of large language models (LLMs) Previous works on in-context example selection for machine translation (MT) focus on superficial word-level features. We propose a syntax-based in-context example selection method for MT, by computing the syntactic similarity between dependency trees.
  arXiv  Detail & Related papers  (2024-03-28T10:13:34Z)
• Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction [8.655807096424732]
  In this paper, we propose a novel ungrammatical-syntax-based in-context example selection strategy for grammatical error correction. Specifically, we measure similarity of sentences based on their syntactic structures with diverse algorithms, and identify optimal ICL examples sharing the most similar ill-formed syntax to the test input.
  arXiv  Detail & Related papers  (2024-03-28T10:05:57Z)
• Contextualization Distillation from Large Language Model for Knowledge Graph Completion [51.126166442122546]
  We introduce the Contextualization Distillation strategy, a plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models to transform compact, structural triplets into context-rich segments. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach.
  arXiv  Detail & Related papers  (2024-01-28T08:56:49Z)
• Towards Effective Disambiguation for Machine Translation with Large Language Models [65.80775710657672]
  We study the capabilities of large language models to translate "ambiguous sentences" Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions.
  arXiv  Detail & Related papers  (2023-09-20T22:22:52Z)
• Alleviating Over-smoothing for Unsupervised Sentence Representation [96.19497378628594]
  We present a Simple method named Self-Contrastive Learning (SSCL) to alleviate this issue. Our proposed method is quite simple and can be easily extended to various state-of-the-art models for performance boosting.
  arXiv  Detail & Related papers  (2023-05-09T11:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.