IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions

• URL: http://arxiv.org/abs/2605.22247v1
• Date: Thu, 21 May 2026 09:53:10 GMT
• Title: IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions
• Abstract summary: We introduce IdioLink, a retrieval benchmark designed to test whether models can link idiomatic expressions to conceptually equivalent meanings expressed in literal or paraphrased forms.<n>IdioLink comprises 10,700 documents and 2,140 queries, spanning 107 idioms with both literal and figurative uses.
• Score: 4.226114812064301
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Idioms pose a fundamental challenge for language models, as their meaning cannot be inferred from surface form alone. Understanding such expressions, therefore, requires semantic abstraction beyond lexical overlap. We introduce IdioLink, a retrieval benchmark designed to test whether models can link idiomatic expressions to conceptually equivalent meanings expressed in literal or paraphrased forms. IdioLink comprises 10,700 documents and 2,140 queries, spanning 107 idioms with both literal and figurative uses. Each document and query is annotated with spans that convey the core meaning. Evaluating strong embedding baselines (e.g., BGE, E5, Contriever, and Qwen), we show that current models struggle to retrieve equivalent meanings across divergent surface realizations, relying instead on topical and shallow semantic cues. IdioLink exposes key gaps in idiom-aware semantic retrieval and provides a challenging testbed for future models.

Related papers

• IdiomX A Multilingual Benchmark for Idiom Understanding, Retrieval, and Interpretation [0.0]
  idiomX is a large-scale multilingual benchmark for idiom understanding, retrieval, and interpretation.<n>The dataset contains over 190K examples spanning 12K+ idioms, with aligned English, Arabic, and French semantic representations.<n>We define a unified four-task benchmark covering idiom detection, context-to-idiom retrieval, Arabic-to-English retrieval, and semantic interpretation.
  arXiv  Detail & Related papers  (2026-04-25T19:54:34Z)
• When Meaning Isn't Literal: Exploring Idiomatic Meaning Across Languages and Modalities [27.883451560702284]
  Hintatic reasoning, deeply intertwined with metaphor and culture, remains a blind spot for contemporary language models.<n>We present Mediom,'' a multilingual, multimodal corpus of 3,533 Hindi, Bengali, and Thai idioms.<n>We benchmark both large language models (textual reasoning) and vision-language models (figurative disambiguation) on Mediom, exposing systematic failures in metaphor comprehension.
  arXiv  Detail & Related papers  (2026-04-12T19:36:00Z)
• Language Modeling and Understanding Through Paraphrase Generation and Detection [4.080540555071174]
  We can express the same thoughts in virtually infinite ways using different words and structures.<n> Modeling paraphrases is a keystone to meaning in computational language models.<n>I propose that decomposing paraphrases into their constituent linguistic aspects offers a more cognitively grounded view of semantic equivalence.
  arXiv  Detail & Related papers  (2026-02-09T05:09:03Z)
• Investigating Idiomaticity in Word Representations [9.208145117062339]
  We focus on noun compounds of varying levels of idiomaticity in two languages (English and Portuguese) We present a dataset of minimal pairs containing human idiomaticity judgments for each noun compound at both type and token levels. We define a set of fine-grained metrics of Affinity and Scaled Similarity to determine how sensitive the models are to perturbations that may lead to changes in idiomaticity.
  arXiv  Detail & Related papers  (2024-11-04T21:05:01Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Idiomatic Expression Identification using Semantic Compatibility [8.355785779504869]
  We study the task of detecting whether a sentence has an idiomatic expression and localizing it. We propose a multi-stage neural architecture with the attention flow mechanism for identifying these expressions. A salient feature of the model is its ability to identify idioms unseen during training with gains from 1.4% to 30.8% over competitive baselines.
  arXiv  Detail & Related papers  (2021-10-19T15:44:28Z)
• Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? [87.20342701232869]
  We investigate the abilities of ungrounded systems to acquire meaning. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation if all expressions in the language are referentially transparent. However, if the language uses non-transparent patterns like variable binding, we show that emulation can become an uncomputable problem.
  arXiv  Detail & Related papers  (2021-04-22T01:00:17Z)
• Speakers Fill Lexical Semantic Gaps with Context [65.08205006886591]
  We operationalise the lexical ambiguity of a word as the entropy of meanings it can take. We find significant correlations between our estimate of ambiguity and the number of synonyms a word has in WordNet. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.
  arXiv  Detail & Related papers  (2020-10-05T17:19:10Z)
• Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressions [87.33156149634392]
  We critically examine RefCOg, a standard benchmark for visual referring expression recognition. We show that 83.7% of test instances do not require reasoning on linguistic structure. We propose two methods, one based on contrastive learning and the other based on multi-task learning, to increase the robustness of ViLBERT.
  arXiv  Detail & Related papers  (2020-05-04T17:09:15Z)
• Word Sense Disambiguation for 158 Languages using Word Embeddings Only [80.79437083582643]
  Disambiguation of word senses in context is easy for humans, but a major challenge for automatic approaches. We present a method that takes as input a standard pre-trained word embedding model and induces a fully-fledged word sense inventory. We use this method to induce a collection of sense inventories for 158 languages on the basis of the original pre-trained fastText word embeddings.
  arXiv  Detail & Related papers  (2020-03-14T14:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.