Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages

• URL: http://arxiv.org/abs/2310.14799v1
• Date: Mon, 23 Oct 2023 10:56:03 GMT
• Title: Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
• Authors: Libo Qin, Qiguang Chen, Fuxuan Wei, Shijue Huang, Wanxiang Che
• Abstract summary: Chain-of-thought (CoT) is capable of eliciting models to explicitly generate reasoning paths. Existing zero-shot prompting techniques are limited to a single language. We introduce cross-lingual prompting (CLP), aiming to improve zero-shot CoT reasoning across languages.
• Score: 46.496557448392494
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain-of-thought (CoT) is capable of eliciting models to explicitly generate reasoning paths, thus promoting reasoning accuracy and attracting increasing attention. Specifically, zero-shot CoT achieves remarkable improvements in a wide range of reasoning tasks by simply instructing the LLM with the prompt "Let's think step by step!". Despite the success of zero-shot CoT, the existing zero-shot prompting techniques remain limited to a single language, making it challenging to generalize to other languages and hindering global development. In this work, we introduce cross-lingual prompting (CLP), aiming to improve zero-shot CoT reasoning across languages. Specifically, CLP consists of two main components: (1) cross-lingual alignment prompting and (2) task-specific solver prompting. The cross-lingual alignment prompting is responsible for aligning representations across different languages, whereas the task-specific solver prompting is used to generate the final chain of thoughts and results for the reasoning task. In addition, we further introduce cross-lingual self-consistent prompting (CLSP) to ensemble different reasoning paths across languages. Our experimental evaluations on several benchmarks demonstrate that CLP and CLSP significantly outperform the existing prompting methods and achieve state-of-the-art performance. We hope this work will inspire further breakthroughs in cross-lingual CoT.

Related papers

• ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language [53.8622516025736]
  We propose a novel end-to-end zero-shot dialogue generation model ChatZero based on cross-lingual code-switching method. Experiments on the multilingual DailyDialog and DSTC7-AVSD datasets demonstrate that ChatZero can achieve more than 90% of the original performance.
  arXiv  Detail & Related papers  (2024-08-16T13:11:53Z)
• xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning [36.34986831526529]
  Chain-of-thought (CoT) has emerged as a powerful technique to elicit reasoning in large language models. We propose a cross-lingual instruction fine-tuning framework (xCOT) to transfer knowledge from high-resource languages to low-resource languages.
  arXiv  Detail & Related papers  (2024-01-13T10:53:53Z)
• Empowering Multi-step Reasoning across Languages via Tree-of-Thoughts [1.8175282137722093]
  Chain-of-Thought (CoT) methods empower Large Language Models (LLMs) to solve complex tasks in a step-by-step manner. The ability to deliver multi-step reasoning remains limited to English because of the imbalance in the distribution of pre-training data. We propose Cross-lingual Tree-of-Thoughts (Cross-ToT), a method for aligning Cross-lingual CoT reasoning across languages.
  arXiv  Detail & Related papers  (2023-11-14T11:49:43Z)
• Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning [98.60739735409243]
  Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks. We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
  arXiv  Detail & Related papers  (2023-04-03T18:46:01Z)
• Multilingual Relation Classification via Efficient and Effective Prompting [9.119073318043952]
  We present the first work on prompt-based multilingual relation classification (RC) We introduce an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages.
  arXiv  Detail & Related papers  (2022-10-25T08:40:23Z)
• Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding [90.87454350016121]
  We develop novel code-switching schemes to generate hard negative examples for contrastive learning at all levels. We develop a label-aware joint model to leverage label semantics for cross-lingual knowledge transfer.
  arXiv  Detail & Related papers  (2022-05-07T13:44:28Z)
• GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding [74.39024160277809]
  We present Global--Local Contrastive Learning Framework (GL-CLeF) to address this shortcoming. Specifically, we employ contrastive learning, leveraging bilingual dictionaries to construct multilingual views of the same utterance. GL-CLeF achieves the best performance and successfully pulls representations of similar sentences across languages closer.
  arXiv  Detail & Related papers  (2022-04-18T13:56:58Z)
• Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval [51.60862829942932]
  We present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved. However, the peak performance is not met using the general-purpose multilingual text encoders off-the-shelf', but rather relying on their variants that have been further specialized for sentence understanding tasks.
  arXiv  Detail & Related papers  (2021-01-21T00:15:38Z)
• On Learning Universal Representations Across Languages [37.555675157198145]
  We extend existing approaches to learn sentence-level representations and show the effectiveness on cross-lingual understanding and generation. Specifically, we propose a Hierarchical Contrastive Learning (HiCTL) method to learn universal representations for parallel sentences distributed in one or multiple languages. We conduct evaluations on two challenging cross-lingual tasks, XTREME and machine translation.
  arXiv  Detail & Related papers  (2020-07-31T10:58:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.