Multilingual Humour-Aware Retrieval with Dense and Re-Ranking Models

• URL: http://arxiv.org/abs/2605.25165v1
• Date: Sun, 24 May 2026 16:50:15 GMT
• Title: Multilingual Humour-Aware Retrieval with Dense and Re-Ranking Models
• Abstract summary: Team DUTH studies multilingual humour-aware information retrieval using the CLEF 2025 JOKER Task 1 benchmark.<n>Our approach combines multilingual XLM-RoBERTa-based dense retrieval with additional system variants, including neural re-ranking.
• Score: 0.4640835690336653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humour-aware information retrieval poses unique challenges beyond standard semantic retrieval, as systems must account not only for topical relevance but also for humour-specific linguistic phenomena such as wordplay, phonetic ambiguity, and polysemy. In this paper, Team DUTH studies multilingual humour-aware information retrieval using the CLEF 2025 JOKER Task 1 benchmark, which evaluates humour retrieval in English and Portuguese. Our approach combines multilingual XLM-RoBERTa-based dense retrieval with additional system variants, including neural re-ranking, in order to assess the extent to which general-purpose Transformer models can capture humour-specific relevance. The results reveal substantial cross-lingual variation. While the Portuguese runs demonstrate comparatively strong performance across MAP, MRR, and early precision metrics, the English runs perform significantly worse, with relevant humorous documents frequently appearing at lower ranks. These findings highlight the limitations of purely semantic dense representations for humour retrieval, particularly when humour depends on surface-level cues that are not explicitly modelled by multilingual encoders. We further analyse contributing factors to this discrepancy, including dataset characteristics, query-document alignment, and variation in humour mechanisms. Overall, the Team DUTH experiments establish multilingual dense-retrieval and re-ranking baselines and provide insights into the challenges of modelling humour-aware relevance within the JOKER framework.

Related papers

• MIMO: Multilingual Information Retrieval via Monolingual Objectives [22.085658351797765]
  Existing embedding models are primarily optimized for Multi-Monolingual retrieval.<n>Milo is a two-stage framework that uses a stable English semantic space from a high-performing teacher model as an anchor.<n>Milo consistently outperforms existing cross-lingual training baselines.
  arXiv  Detail & Related papers  (2026-05-29T11:34:15Z)
• Improving Semantic Proximity in Information Retrieval through Cross-Lingual Alignment [26.443216883777882]
  Cross-Lingual Information Retrieval (CLIR) has emerged as an important research area.<n>Most multilingual retrievers tend to prioritize unrelated English documents over the related document written in the same language as the query.<n>We propose a novel training strategy aimed at enhancing cross-lingual alignment.
  arXiv  Detail & Related papers  (2026-04-07T10:35:52Z)
• Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation [11.110312833458421]
  We study output language drift in multilingual RAG across multiple datasets, languages, and LLM backbones.<n>Our experiments reveal that the drift results from decoder-level collapse, where dominant token distributions and high-frequency English patterns dominate the intended generation language.<n>We propose Soft Constrained Decoding (SCD), a lightweight, training-free decoding strategy that gently steers generation toward the target language.
  arXiv  Detail & Related papers  (2025-11-13T05:36:31Z)
• Investigating Language Preference of Multilingual RAG Systems [4.438698005789677]
  MRAG systems struggle with retrieving relevant information due to linguistic variations between queries and documents.<n>We propose Dual Knowledge Multilingual RAG (DKM-RAG), a framework that fuses translated multilingual passages with complementary model knowledge.<n> Empirical results demonstrate that DKM-RAG mitigates language preference in generation and enhances performance across diverse linguistic settings.
  arXiv  Detail & Related papers  (2025-02-16T15:54:05Z)
• Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples [29.62231663945077]
  We introduce Cross-Lingual Semantic Discrimination (D), a lightweight evaluation task that requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors.<n> CLSD measures an embedding model's ability to rank the true parallel sentence above semantically misleading but lexically similar alternatives.<n>Our experiments show that models fine-tuned for retrieval tasks benefit from pivoting through English, whereas bitext mining models perform best in direct cross-lingual settings.
  arXiv  Detail & Related papers  (2025-02-12T18:54:37Z)
• mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval [61.17793165194077]
  We introduce mFollowIR, a benchmark for measuring instruction-following ability in retrieval models.<n>We present results for both multilingual (XX-XX) and cross-lingual (En-XX) performance.<n>We see strong cross-lingual performance with English-based retrievers that trained using instructions, but find a notable drop in performance in the multilingual setting.
  arXiv  Detail & Related papers  (2025-01-31T16:24:46Z)
• P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs [84.24644520272835]
  We introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets.<n>P-MMEval delivers consistent language coverage across various datasets and provides parallel samples.<n>We conduct extensive experiments on representative multilingual model series to compare performances across models and tasks.
  arXiv  Detail & Related papers  (2024-11-14T01:29:36Z)
• On Cross-Lingual Retrieval with Multilingual Text Encoders [51.60862829942932]
  We study the suitability of state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. We benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR experiments. We evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we learn to rank) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments.
  arXiv  Detail & Related papers  (2021-12-21T08:10:27Z)
• Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations [14.852199996061287]
  We develop a Hindi-English code-mixed dataset, MaSaC, for the multi-modal sarcasm detection and humor classification in conversational dialog. We propose MSH-COMICS, a novel attention-rich neural architecture for the utterance classification.
  arXiv  Detail & Related papers  (2021-05-20T18:33:55Z)
• Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval [51.60862829942932]
  We present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved. However, the peak performance is not met using the general-purpose multilingual text encoders off-the-shelf', but rather relying on their variants that have been further specialized for sentence understanding tasks.
  arXiv  Detail & Related papers  (2021-01-21T00:15:38Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)
• Predicting the Humorousness of Tweets Using Gaussian Process Preference Learning [56.18809963342249]
  We present a probabilistic approach that learns to rank and rate the humorousness of short texts by exploiting human preference judgments and automatically sourced linguistic annotations. We report system performance for the campaign's two subtasks, humour detection and funniness score prediction, and discuss some issues arising from the conversion between the numeric scores used in the HAHA@IberLEF 2019 data and the pairwise judgment annotations required for our method.
  arXiv  Detail & Related papers  (2020-08-03T13:05:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.