Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models

• URL: http://arxiv.org/abs/2308.13961v2
• Date: Mon, 25 Dec 2023 02:54:13 GMT
• Title: Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models
• Authors: Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Yang, Shimin Tao, Yanghua Xiao
• Abstract summary: idioms, with their non-compositional nature, pose particular challenges for Transformer-based systems. Traditional methods, which replace idioms using existing knowledge bases (KBs), often lack scale and context awareness. We introduce a multilingual idiom KB (IdiomKB) developed using large LMs to address this. This KB facilitates better translation by smaller models, such as BLOOMZ (7.1B), Alpaca (7B), and InstructGPT (6.7B)
• Score: 57.60487455727155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To translate well, machine translation (MT) systems and general-purposed language models (LMs) need a deep understanding of both source and target languages and cultures. Therefore, idioms, with their non-compositional nature, pose particular challenges for Transformer-based systems, as literal translations often miss the intended meaning. Traditional methods, which replace idioms using existing knowledge bases (KBs), often lack scale and context awareness. Addressing these challenges, our approach prioritizes context awareness and scalability, allowing for offline storage of idioms in a manageable KB size. This ensures efficient serving with smaller models and provides a more comprehensive understanding of idiomatic expressions. We introduce a multilingual idiom KB (IdiomKB) developed using large LMs to address this. This KB facilitates better translation by smaller models, such as BLOOMZ (7.1B), Alpaca (7B), and InstructGPT (6.7B), by retrieving idioms' figurative meanings. We present a novel, GPT-4-powered metric for human-aligned evaluation, demonstrating that IdiomKB considerably boosts model performance. Human evaluations further validate our KB's quality.

Related papers

• KBLaM: Knowledge Base augmented Language Model [8.247901935078357]
  We propose Knowledge Base augmented Language Model (KBLaM) for augmenting Large Language Models with external knowledge. KBLaM works with a knowledge base constructed from a corpus of documents, transforming each piece of knowledge in the KB into continuous key-value vector pairs. Experiments demonstrate KBLaM's effectiveness in various tasks, including question-answering and open-ended reasoning.
  arXiv  Detail & Related papers  (2024-10-14T12:45:10Z)
• TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases [20.751369684593985]
  TIARA outperforms previous SOTA, including those using PLMs or oracle entity annotations, by at least 4.1 and 1.1 F1 points on GrailQA and WebQuestionsSP.
  arXiv  Detail & Related papers  (2022-10-24T02:41:10Z)
• Improving Multilingual Neural Machine Translation System for Indic Languages [0.0]
  We propose a multilingual neural machine translation (MNMT) system to address the issues related to low-resource language translation. A state-of-the-art transformer architecture is used to realize the proposed model. Trials over a good amount of data reveal its superiority over the conventional models.
  arXiv  Detail & Related papers  (2022-09-27T09:51:56Z)
• The Analysis about Building Cross-lingual Sememe Knowledge Base Based on Deep Clustering Network [0.7310043452300736]
  Sememe knowledge bases (KBs) contain words annotated with sememes. We propose an unsupervised method based on a deep clustering network (DCN) to build a sememe KB.
  arXiv  Detail & Related papers  (2022-08-10T17:40:45Z)
• A Review on Language Models as Knowledge Bases [55.035030134703995]
  Recently, there has been a surge of interest in the NLP community on the use of pretrained Language Models (LMs) as Knowledge Bases (KBs)
  arXiv  Detail & Related papers  (2022-04-12T18:35:23Z)
• Prix-LM: Pretraining for Multilingual Knowledge Base Construction [59.02868906044296]
  We propose a unified framework, Prix-LM, for multilingual knowledge construction and completion. We leverage two types of knowledge, monolingual triples and cross-lingual links, extracted from existing multilingual KBs. Experiments on standard entity-related tasks, such as link prediction in multiple languages, cross-lingual entity linking and bilingual lexicon induction, demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2021-10-16T02:08:46Z)
• Automatic Construction of Sememe Knowledge Bases via Dictionaries [53.8700954466358]
  Sememe knowledge bases (SKBs) enable sememes to be applied to natural language processing. Most languages have no SKBs, and manual construction of SKBs is time-consuming and labor-intensive. We propose a simple and fully automatic method of building an SKB via an existing dictionary.
  arXiv  Detail & Related papers  (2021-05-26T14:41:01Z)
• Reasoning Over Virtual Knowledge Bases With Open Predicate Relations [85.19305347984515]
  We present the Open Predicate Query Language (OPQL) OPQL is a method for constructing a virtual Knowledge Base (VKB) trained entirely from text. We demonstrate that OPQL outperforms prior VKB methods on two different KB reasoning tasks.
  arXiv  Detail & Related papers  (2021-02-14T01:29:54Z)
• Improving Candidate Generation for Low-resource Cross-lingual Entity Linking [81.41804263432684]
  Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts. In this paper, we propose three improvements that (1) reduce the disconnect between entity mentions and KB entries, and (2) improve the robustness of the model to low-resource scenarios.
  arXiv  Detail & Related papers  (2020-03-03T05:32:09Z)
• Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base [34.837700505583]
  We describe a novel way of representing a symbolic knowledge base (KB) called a sparse-matrix reified KB. This representation enables neural modules that are fully differentiable, faithful to the original semantics of the KB, expressive enough to model multi-hop inferences, and scalable enough to use with realistically large KBs.
  arXiv  Detail & Related papers  (2020-02-14T16:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.