The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models

• URL: http://arxiv.org/abs/2406.19358v1
• Date: Thu, 27 Jun 2024 17:38:45 GMT
• Title: The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models
• Authors: Xiliang Zhu, Shayna Gardiner, Tere Roldán, David Rossouw,
• Abstract summary: This study compares the cross-lingual transfer capability of public Small Language Models (M) and Large Language Models (LLM) Our findings reveal that among public models, SMLMs exhibit superior zero-shot cross-lingual performance. In few-shot cross-lingual settings, public LLMs demonstrate an enhanced adaptive potential.
• Score: 0.4821250031784094
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Sentiment analysis serves as a pivotal component in Natural Language Processing (NLP). Advancements in multilingual pre-trained models such as XLM-R and mT5 have contributed to the increasing interest in cross-lingual sentiment analysis. The recent emergence in Large Language Models (LLM) has significantly advanced general NLP tasks, however, the capability of such LLMs in cross-lingual sentiment analysis has not been fully studied. This work undertakes an empirical analysis to compare the cross-lingual transfer capability of public Small Multilingual Language Models (SMLM) like XLM-R, against English-centric LLMs such as Llama-3, in the context of sentiment analysis across English, Spanish, French and Chinese. Our findings reveal that among public models, SMLMs exhibit superior zero-shot cross-lingual performance relative to LLMs. However, in few-shot cross-lingual settings, public LLMs demonstrate an enhanced adaptive potential. In addition, we observe that proprietary GPT-3.5 and GPT-4 lead in zero-shot cross-lingual capability, but are outpaced by public models in few-shot scenarios.

Related papers

• Preference Tuning For Toxicity Mitigation Generalizes Across Languages [17.784213168942117]
  This work explores zero-shot cross-lingual generalization of preference tuning in multilingual Large Language Models. We demonstrate that Direct Preference Optimization training with only English data can significantly reduce toxicity in multilingual open-ended generations.
  arXiv  Detail & Related papers  (2024-06-23T22:53:47Z)
• Low-Resource Cross-Lingual Summarization through Few-Shot Learning with Large Language Models [4.9325206373289125]
  Cross-lingual summarization (XLS) aims to generate a summary in a target language different from the source language document. Large language models (LLMs) have shown promising zero-shot XLS performance, their few-shot capabilities on this task remain unexplored. We investigate the few-shot XLS performance of various models, including Mistral-7B-Instruct-v0.2, GPT-3.5, and GPT-4.
  arXiv  Detail & Related papers  (2024-06-07T04:31:41Z)
• Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean [3.4735169184479524]
  Large language models (LLMs) use pretraining to predict the subsequent word; however, their expansion requires significant computing resources. This study proposed three strategies to enhance the performance of LRLs based on the publicly available MLLMs.
  arXiv  Detail & Related papers  (2024-03-16T10:26:38Z)
• Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet? [82.02076369811402]
  Supervised fine-tuning (SFT), supervised instruction tuning (SIT) and in-context learning (ICL) are three alternative, de facto standard approaches to few-shot learning. We present an extensive and systematic comparison of the three approaches, testing them on 6 high- and low-resource languages, three different NLU tasks, and a myriad of language and domain setups. Our observations show that supervised instruction tuning has the best trade-off between performance and resource requirements.
  arXiv  Detail & Related papers  (2024-03-04T10:48:13Z)
• Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models [117.20416338476856]
  Large language models (LLMs) demonstrate remarkable multilingual capabilities without being pre-trained on specially curated multilingual parallel corpora. We propose a novel detection method, language activation probability entropy (LAPE), to identify language-specific neurons within LLMs. Our findings indicate that LLMs' proficiency in processing a particular language is predominantly due to a small subset of neurons.
  arXiv  Detail & Related papers  (2024-02-26T09:36:05Z)
• Unveiling Linguistic Regions in Large Language Models [49.298360366468934]
  Large Language Models (LLMs) have demonstrated considerable cross-lingual alignment and generalization ability. This paper conducts several investigations on the linguistic competence of LLMs.
  arXiv  Detail & Related papers  (2024-02-22T16:56:13Z)
• Zero-Shot Cross-Lingual Reranking with Large Language Models for Low-Resource Languages [51.301942056881146]
  We investigate how large language models (LLMs) function as rerankers in cross-lingual information retrieval systems for African languages. Our implementation covers English and four African languages (Hausa, Somali, Swahili, and Yoruba) We examine cross-lingual reranking with queries in English and passages in the African languages.
  arXiv  Detail & Related papers  (2023-12-26T18:38:54Z)
• MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks [12.665447518524187]
  This study aims to perform a thorough evaluation of the non-English capabilities of SoTA LLMs by comparing them on the same set of multilingual datasets. Our benchmark comprises 22 datasets covering 83 languages, including low-resource African languages. We also perform a study on data contamination and find that several models are likely to be contaminated with multilingual evaluation benchmarks.
  arXiv  Detail & Related papers  (2023-11-13T16:45:37Z)
• SCALE: Synergized Collaboration of Asymmetric Language Translation Engines [105.8983433641208]
  We introduce a collaborative framework that connects compact Specialized Translation Models (STMs) and general-purpose Large Language Models (LLMs) as one unified translation engine. By introducing translation from STM into the triplet in-context demonstrations, SCALE unlocks refinement and pivoting ability of LLM. Our experiments show that SCALE significantly outperforms both few-shot LLMs (GPT-4) and specialized models (NLLB) in challenging low-resource settings.
  arXiv  Detail & Related papers  (2023-09-29T08:46:38Z)
• Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysis [6.471458199049549]
  In this study, we present a sizeable manually annotated dataset encompassing 33,606 Bangla news tweets and Facebook comments. We also investigate zero- and few-shot in-context learning with several language models, including Flan-T5, GPT-4, and Bloomz. Our findings suggest that monolingual transformer-based models consistently outperform other models, even in zero and few-shot scenarios.
  arXiv  Detail & Related papers  (2023-08-21T15:19:10Z)
• Zero-Shot Cross-Lingual Summarization via Large Language Models [108.30673793281987]
  Cross-lingual summarization ( CLS) generates a summary in a different target language. Recent emergence of Large Language Models (LLMs) has attracted wide attention from the computational linguistics community. In this report, we empirically use various prompts to guide LLMs to perform zero-shot CLS from different paradigms.
  arXiv  Detail & Related papers  (2023-02-28T01:27:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.