Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities

• URL: http://arxiv.org/abs/2605.21369v1
• Date: Wed, 20 May 2026 16:35:09 GMT
• Title: Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities
• Abstract summary: This paper describes the fifth edition of the Shared Task on Multilingual Coreference Resolution, held in conjunction with CODI-CRAC 2026.<n>The 2026 edition specifically emphasizes long-range entities, defined as coreferential chains spanning significant distances, across many words and sentences.<n>The task expanded its linguistic scope by incorporating five new datasets and two additional languages.
• Score: 1.5851688800400288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes the fifth edition of the Shared Task on Multilingual Coreference Resolution, held in conjunction with the CODI-CRAC 2026 workshop. Building on previous iterations, the task required participants to develop systems capable of mention identification and identity-based coreference clustering. The 2026 edition specifically emphasizes long-range entities, defined as coreferential chains spanning significant distances, across many words and sentences. The task expanded its linguistic scope by incorporating five new datasets and two additional languages. These additions leverage version 1.4 of CorefUD, a harmonized multilingual collection comprising 27 datasets in 19 languages. In total, ten systems participated, including four LLM-based approaches (three fine-tuned models and one few-shot approach). While traditional systems still maintained their lead, LLMs demonstrated significant potential, suggesting they may soon challenge established approaches in future editions.

Related papers

• Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models [19.79187126303009]
  In 2026, we held the fourth iteration of the SHROOM-Visions task series: SHROOM-Visions (textbfShared-task on textbfHallucinations and textbfRelated textbfMistakes in textbfVision language modeltextbfs)<n>This time we aim to tackle hallucinations through a model-agnostic detection task focused on large vision-language models.<n>The evaluation uses a five-class taxonomy of hallucinations spanning four languages: Chinese, English, French
  arXiv  Detail & Related papers  (2026-08-26T11:44:35Z)
• Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026 [57.48615919717867]
  We present KIT's submission to the Long and Short Instruction Following tracks in the unconstrained setting.<n>Our approach combines a general data augmentation pipeline that converts short-form corpora into long-form training data.<n>We show that likelihood-based re-ranking, while highly effective for ASR, systematically degrades semantic tasks.
  arXiv  Detail & Related papers  (2026-06-03T11:13:37Z)
• Findings of the Fourth Shared Task on Multilingual Coreference Resolution: Can LLMs Dethrone Traditional Approaches? [1.5851688800400288]
  The paper presents an overview of the fourth edition of the Shared Task on Multilingual Coreference Resolution.<n>As in the previous editions, participants were challenged to develop systems that identify mentions and cluster them according to identity coreference.<n>A key innovation of this year's task was the introduction of a dedicated Large Language Model track.
  arXiv  Detail & Related papers  (2025-09-22T13:52:32Z)
• LUSIFER: Language Universal Space Integration for Enhanced Multilingual Embeddings with Large Language Models [89.13128402847943]
  We present LUSIFER, a novel zero-shot approach that adapts LLM-based embedding models for multilingual tasks without requiring multilingual supervision.<n>LUSIFER's architecture combines a multilingual encoder, serving as a language-universal learner, with an LLM-based embedding model optimized for embedding-specific tasks.<n>We introduce a new benchmark encompassing 5 primary embedding tasks, 123 diverse datasets, and coverage across 14 languages.
  arXiv  Detail & Related papers  (2025-01-01T15:43:07Z)
• Findings of the Third Shared Task on Multilingual Coreference Resolution [1.4191726968556324]
  The paper presents an overview of the third edition of the shared task on multilingual coreference resolution, held as part of the CRAC 2024 workshop. Similar to the previous two editions, the participants were challenged to develop systems capable of identifying mentions and clustering them based on identity coreference. This year's edition took another step towards real-world application by not providing participants with gold slots for zero anaphora, increasing the task's complexity and realism.
  arXiv  Detail & Related papers  (2024-10-21T12:30:44Z)
• EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models [50.459861376459656]
  EMMA-500 is a large-scale multilingual language model continue-trained on texts across 546 languages.<n>Our results highlight the effectiveness of continual pre-training in expanding large language models' language capacity.
  arXiv  Detail & Related papers  (2024-09-26T14:40:45Z)
• SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes [48.83290963506378]
  This paper presents the results of the SHROOM, a shared task focused on detecting hallucinations. We observe a number of key trends in how this approach was tackled. While a majority of the teams did outperform our proposed baseline system, the performances of top-scoring systems are still consistent with a random handling of the more challenging items.
  arXiv  Detail & Related papers  (2024-03-12T15:06:22Z)
• Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyond [87.4049283495551]
  The 2023 Multilingual Speech Universal Performance Benchmark (ML-SUPERB) Challenge expands upon the acclaimed SUPERB framework.<n>The challenge garnered 12 model submissions and 54 language corpora, resulting in a comprehensive benchmark encompassing 154 languages.<n>The findings indicate that merely scaling models is not the definitive solution for multilingual speech tasks.
  arXiv  Detail & Related papers  (2023-10-09T08:30:01Z)
• OLR 2021 Challenge: Datasets, Rules and Baselines [23.878103387338918]
  The data profile, four tasks, two baselines, and the evaluation principles are introduced in this paper. In addition to the Language Identification (LID) tasks, multilingual Automatic Speech Recognition (ASR) tasks are introduced to OLR 2021 Challenge for the first time.
  arXiv  Detail & Related papers  (2021-07-23T09:57:29Z)
• CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP [68.2650714613869]
  We propose a data augmentation framework to generate multi-lingual code-switching data to fine-tune mBERT. Compared with the existing work, our method does not rely on bilingual sentences for training, and requires only one training process for multiple target languages.
  arXiv  Detail & Related papers  (2020-06-11T13:15:59Z)
• XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation [100.09099800591822]
  XGLUE is a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models. XGLUE provides 11 diversified tasks that cover both natural language understanding and generation scenarios.
  arXiv  Detail & Related papers  (2020-04-03T07:03:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.