USTCCTSU at SemEval-2024 Task 1: Reducing Anisotropy for Cross-lingual Semantic Textual Relatedness Task

• URL: http://arxiv.org/abs/2411.18990v1
• Date: Thu, 28 Nov 2024 08:40:14 GMT
• Title: USTCCTSU at SemEval-2024 Task 1: Reducing Anisotropy for Cross-lingual Semantic Textual Relatedness Task
• Authors: Jianjian Li, Shengwei Liang, Yong Liao, Hongping Deng, Haiyang Yu,
• Abstract summary: Cross-lingual semantic textual relatedness task is an important research task that addresses challenges in cross-lingual communication and text understanding. It helps establish semantic connections between different languages, crucial for downstream tasks like machine translation, multilingual information retrieval, and cross-lingual text understanding. With our approach, we achieve a 2nd score in Spanish, a 3rd in Indonesian, and multiple entries in the top ten results in the competition's track C.
• Score: 17.905282052666333
• License:
• Abstract: Cross-lingual semantic textual relatedness task is an important research task that addresses challenges in cross-lingual communication and text understanding. It helps establish semantic connections between different languages, crucial for downstream tasks like machine translation, multilingual information retrieval, and cross-lingual text understanding.Based on extensive comparative experiments, we choose the XLM-R-base as our base model and use pre-trained sentence representations based on whitening to reduce anisotropy.Additionally, for the given training data, we design a delicate data filtering method to alleviate the curse of multilingualism. With our approach, we achieve a 2nd score in Spanish, a 3rd in Indonesian, and multiple entries in the top ten results in the competition's track C. We further do a comprehensive analysis to inspire future research aimed at improving performance on cross-lingual tasks.

Related papers

• CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment [38.35458193262633]
  English-centric models are usually suboptimal in other languages. We propose a novel approach called CrossIn, which utilizes a mixed composition of cross-lingual instruction tuning data.
  arXiv  Detail & Related papers  (2024-04-18T06:20:50Z)
• AAdaM at SemEval-2024 Task 1: Augmentation and Adaptation for Multilingual Semantic Textual Relatedness [16.896143197472114]
  This paper presents our system developed for the SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian languages. We propose using machine translation for data augmentation to address the low-resource challenge of limited training data. We achieve competitive results in the shared task: our system performs the best among all ranked teams in both subtask A (supervised learning) and subtask C (cross-lingual transfer)
  arXiv  Detail & Related papers  (2024-04-01T21:21:15Z)
• Cross-lingual Contextualized Phrase Retrieval [63.80154430930898]
  We propose a new task formulation of dense retrieval, cross-lingual contextualized phrase retrieval. We train our Cross-lingual Contextualized Phrase Retriever (CCPR) using contrastive learning. On the phrase retrieval task, CCPR surpasses baselines by a significant margin, achieving a top-1 accuracy that is at least 13 points higher.
  arXiv  Detail & Related papers  (2024-03-25T14:46:51Z)
• Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning [98.60739735409243]
  Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks. We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
  arXiv  Detail & Related papers  (2023-04-03T18:46:01Z)
• CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval [73.48591773882052]
  Most fact-checking approaches focus on English only due to the data scarcity issue in other languages. We present the first fact-checking framework augmented with crosslingual retrieval. We train the retriever with our proposed Crosslingual Inverse Cloze Task (XICT)
  arXiv  Detail & Related papers  (2022-09-05T17:36:14Z)
• Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation [80.16548523140025]
  We extend the vanilla pretrain-finetune pipeline with extra code-switching restore task to bridge the gap between the pretrain and finetune stages. Our approach could narrow the cross-lingual sentence representation distance and improve low-frequency word translation with trivial computational cost.
  arXiv  Detail & Related papers  (2022-04-16T16:08:38Z)
• Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task [26.703809355057224]
  We conduct a detailed analysis to understand the impact of the auxiliary task on the primary task within the multitask learning framework. Our analysis confirms that multitask learning tends to generate similar decoder representations from different modalities. Inspired by these findings, we propose three methods to improve translation quality.
  arXiv  Detail & Related papers  (2021-07-12T23:53:40Z)
• MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers [1.869621561196521]
  We present our approach for solving the SemEval 2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC) The goal is to detect whether a given word common to both the sentences evokes the same meaning. We submit systems for both the settings - Multilingual and Cross-Lingual.
  arXiv  Detail & Related papers  (2021-04-04T08:49:28Z)
• Mixed-Lingual Pre-training for Cross-lingual Summarization [54.4823498438831]
  Cross-lingual Summarization aims at producing a summary in the target language for an article in the source language. We propose a solution based on mixed-lingual pre-training that leverages both cross-lingual tasks like translation and monolingual tasks like masked language models. Our model achieves an improvement of 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over state-of-the-art results.
  arXiv  Detail & Related papers  (2020-10-18T00:21:53Z)
• Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension [86.1617182312817]
  We propose two auxiliary tasks in the fine-tuning stage to create additional phrase boundary supervision. A mixed Machine Reading task, which translates the question or passage to other languages and builds cross-lingual question-passage pairs. A language-agnostic knowledge masking task by leveraging knowledge phrases mined from web.
  arXiv  Detail & Related papers  (2020-04-29T10:44:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.