The SIGMORPHON 2022 Shared Task on Morpheme Segmentation

• URL: http://arxiv.org/abs/2206.07615v1
• Date: Wed, 15 Jun 2022 15:57:22 GMT
• Title: The SIGMORPHON 2022 Shared Task on Morpheme Segmentation
• Abstract summary: The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to decompose a word into a sequence of morphemes. The best systems outperformed all three state-of-the-art subword tokenization methods (BPE, ULM, Morfessor2) by 30.71% absolute. To facilitate error analysis and support any type of future studies, we released all system predictions, the evaluation script, and all gold standard datasets.
• Score: 39.44280269663147
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to decompose a word into a sequence of morphemes and covered most types of morphology: compounds, derivations, and inflections. Subtask 1, word-level morpheme segmentation, covered 5 million words in 9 languages (Czech, English, Spanish, Hungarian, French, Italian, Russian, Latin, Mongolian) and received 13 system submissions from 7 teams and the best system averaged 97.29% F1 score across all languages, ranging English (93.84%) to Latin (99.38%). Subtask 2, sentence-level morpheme segmentation, covered 18,735 sentences in 3 languages (Czech, English, Mongolian), received 10 system submissions from 3 teams, and the best systems outperformed all three state-of-the-art subword tokenization methods (BPE, ULM, Morfessor2) by 30.71% absolute. To facilitate error analysis and support any type of future studies, we released all system predictions, the evaluation script, and all gold standard datasets.

Related papers

• Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering [59.6753725406784]
  FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence.<n>The final-test set contains 256 items, split evenly between easy and expert tiers.<n>The strongest systems are closely clustered, with the top four separated by less than one percentage point in ROUGE-1 F1.
  arXiv  Detail & Related papers  (2026-07-22T07:54:17Z)
• Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering [59.67537254067838]
  FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi.<n>The final-test set contains 800 questions, with 200 questions per language; gold answers were withheld during submission, and each language was ranked independently by accuracy.<n>Top accuracies range from 92.0% in Hindi to 97.5% in English and Arabic, with the same leading teams appearing near the top across all four languages.
  arXiv  Detail & Related papers  (2026-07-22T07:43:44Z)
• PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation [0.0]
  We present our system for SemEval-2026 Task 9: Multilingual Polarization Detection, a binary classification task spanning 22 languages.<n>Our approach fine-tunes separate Gemma3 models per language using Low-Rank Adaptation (LoRA), augmented with synthetic data generated by a large language model (LLM)<n>Our final system achieves a mean macro-F1 of 0.811 across all 22 languages, ranking 2nd overall of the participating teams, with 1st place finishes in 3 languages and top-3 in 8 languages.
  arXiv  Detail & Related papers  (2026-05-06T17:29:14Z)
• MKJ at SemEval-2026 Task 9: A Comparative Study of Generalist, Specialist, and Ensemble Strategies for Multilingual Polarization [0.0]
  We present a systematic study of multilingual polarization detection across 22 languages for SemEval-2026 Task 9 (Subtask 1).<n>Rather than enforcing a single universal architecture, we adopt a language-adaptive framework that switches between multilingual generalists, language-specific specialists, and hybrid ensembles based on development performance.
  arXiv  Detail & Related papers  (2026-04-23T07:34:26Z)
• LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them? [3.809788214307542]
  We identify and characterize two key failure modes: the multilingual transfer bottleneck and the language consistency bottleneck.<n>We extend logit lens analysis to track language probabilities layer by layer and compute cross-lingual semantic similarity of hidden states.<n>This is the first approach to leverage layer-localization of language control for efficient multilingual adaptation.
  arXiv  Detail & Related papers  (2026-01-27T19:38:12Z)
• AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization [0.5274891943689054]
  Claim normalization is a crucial step in automated fact-checking pipelines.<n>This paper details our submission to the CLEF-2025 CheckThat! Task2, which challenges systems to perform claim normalization across twenty languages.
  arXiv  Detail & Related papers  (2025-09-15T01:19:49Z)
• A Joint Multitask Model for Morpho-Syntactic Parsing [2.265421354393977]
  We present a joint multitask model for the UniDive 2025 Morpho-Syntactic Parsing shared task.<n>Our system uses a shared XLM-RoBERTa encoder with three specialized decoders for content word identification, dependency parsing, and morphosyntactic feature prediction.
  arXiv  Detail & Related papers  (2025-08-19T23:09:36Z)
• SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection [76.18321723846616]
  Task covers more than 30 languages from seven distinct language families. Data instances are multi-labeled with six emotional classes, with additional datasets in 11 languages annotated for emotion intensity. Participants were asked to predict labels in three tracks: (a) multilabel emotion detection, (b) emotion intensity score detection, and (c) cross-lingual emotion detection.
  arXiv  Detail & Related papers  (2025-03-10T12:49:31Z)
• SmurfCat at PAN 2024 TextDetox: Alignment of Multilingual Transformers for Text Detoxification [41.94295877935867]
  This paper presents a solution for the Multilingual Text Detoxification task in the PAN-2024 competition of the SmurfCat team. Using data augmentation through machine translation and a special filtering procedure, we collected an additional multilingual parallel dataset for text detoxification. We fine-tuned several multilingual sequence-to-sequence models, such as mT0 and Aya, on a text detoxification task.
  arXiv  Detail & Related papers  (2024-07-07T17:19:34Z)
• The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants [80.4837840962273]
  We present Belebele, a dataset spanning 122 language variants. This dataset enables the evaluation of text models in high-, medium-, and low-resource languages.
  arXiv  Detail & Related papers  (2023-08-31T17:43:08Z)
• SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation [52.186343500576214]
  We introduce SEAHORSE, a dataset for multilingual, multifaceted summarization evaluation. SEAHORSE consists of 96K summaries with human ratings along 6 dimensions of text quality. We show that metrics trained with SEAHORSE achieve strong performance on the out-of-domain meta-evaluation benchmarks TRUE and mFACE.
  arXiv  Detail & Related papers  (2023-05-22T16:25:07Z)
• MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages [54.002969723086075]
  We evaluate cross-lingual open-retrieval question answering systems in 16 typologically diverse languages. The best system leveraging iteratively mined diverse negative examples achieves 32.2 F1, outperforming our baseline by 4.5 points. The second best system uses entity-aware contextualized representations for document retrieval, and achieves significant improvements in Tamil (20.8 F1), whereas most of the other systems yield nearly zero scores.
  arXiv  Detail & Related papers  (2022-07-02T06:54:10Z)
• 1Cademy at Semeval-2022 Task 1: Investigating the Effectiveness of Multilingual, Multitask, and Language-Agnostic Tricks for the Reverse Dictionary Task [13.480318097164389]
  We focus on the Reverse Dictionary Track of the SemEval2022 task of matching dictionary glosses to word embeddings. Models convert the input of sentences to three types of embeddings: SGNS, Char, and Electra. Our proposed Elmobased monolingual model achieves the highest outcome.
  arXiv  Detail & Related papers  (2022-06-08T06:39:04Z)
• UniMorph 4.0: Universal Morphology [104.69846084893298]
  This paper presents the expansions and improvements made on several fronts over the last couple of years. Collaborative efforts by numerous linguists have added 67 new languages, including 30 endangered languages. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages.
  arXiv  Detail & Related papers  (2022-05-07T09:19:02Z)
• SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection [81.85463892070085]
  The SIGMORPHON 2020 task on morphological reinflection aims to investigate systems' ability to generalize across typologically distinct languages. Systems were developed using data from 45 languages and just 5 language families, fine-tuned with data from an additional 45 languages and 10 language families (13 in total), and evaluated on all 90 languages.
  arXiv  Detail & Related papers  (2020-06-20T13:24:14Z)
• The SIGMORPHON 2020 Shared Task on Unsupervised Morphological Paradigm Completion [28.728844366333185]
  In this paper, we describe the findings of the SIGMORPHON 2020 shared task on unsupervised morphological paradigm completion. Participants were asked to submit systems which take raw text and a list of lemmas as input, and output all inflected forms. We present an analysis here, so that this shared task will ground further research on the topic.
  arXiv  Detail & Related papers  (2020-05-28T03:09:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.