Tug-of-war between idiom's figurative and literal meanings in LLMs

• URL: http://arxiv.org/abs/2506.01723v3
• Date: Fri, 06 Jun 2025 13:41:57 GMT
• Title: Tug-of-war between idiom's figurative and literal meanings in LLMs
• Authors: Soyoung Oh, Xinting Huang, Mathis Pink, Michael Hahn, Vera Demberg,
• Abstract summary: Non-compositional figurative meanings often strongly diverge from the idiom's literal interpretation.<n>This duality requires a model to learn representing and deciding between the two meanings to interpret an idiom in a figurative sense, or literally.<n>We use tools from mechanistic interpretability to trace how a large pretrained causal transformer deals with this ambiguity.
• Score: 16.986454464806616
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Idioms present a unique challenge for language models due to their non-compositional figurative meanings, which often strongly diverge from the idiom's literal interpretation. This duality requires a model to learn representing and deciding between the two meanings to interpret an idiom in a figurative sense, or literally. In this paper, we employ tools from mechanistic interpretability to trace how a large pretrained causal transformer (LLama3.2-1B-base) deals with this ambiguity. We localize three steps of idiom processing: First, the idiom's figurative meaning is retrieved in early attention and MLP sublayers. We identify specific attention heads which boost the figurative meaning of the idiom while suppressing the idiom's literal interpretation. The model subsequently represents the figurative representation through an intermediate path. Meanwhile, a parallel bypass route forwards literal interpretation, ensuring that a both reading remain available. Overall, our findings provide a mechanistic evidence for idiom comprehension in an autoregressive transformer.

Related papers

• Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models [4.946483489399819]
  Large Language Models (LLMs) are prone to hallucination, the generation of factually incorrect statements.<n>This work investigates the intrinsic, architectural origins of this failure mode through three primary contributions.
  arXiv  Detail & Related papers  (2025-10-07T16:40:31Z)
• Meanings are like Onions: a Layered Approach to Metaphor Processing [0.0]
  We propose a model of metaphor processing that treats meaning as an onion.<n>At the first level, metaphors are annotated through basic conceptual elements.<n>At the second level, we model conceptual combinations, linking components to emergent meanings.<n>At the third level, we introduce a pragmatic vocabulary to capture speaker intent, communicative function, and contextual effects.
  arXiv  Detail & Related papers  (2025-07-14T14:56:46Z)
• How do Transformers Learn Implicit Reasoning? [67.02072851088637]
  We study how implicit multi-hop reasoning emerges by training transformers from scratch in a controlled symbolic environment.<n>We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures.
  arXiv  Detail & Related papers  (2025-05-29T17:02:49Z)
• SlangDIT: Benchmarking LLMs in Interpretative Slang Translation [89.48208612476068]
  This paper introduces the interpretative slang translation task (named SlangDIT)<n>It consists of three sub-tasks: slang detection, cross-lingual slang explanation, and slang translation within the current context.<n>Based on the benchmark, we propose a deep thinking model, named SlangOWL. It firstly identifies whether the sentence contains a slang, and then judges whether the slang is polysemous and analyze its possible meaning.
  arXiv  Detail & Related papers  (2025-05-20T10:37:34Z)
• Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attention [11.073959609358088]
  We investigate the processing of garden-path sentences and the fate of lingering misinterpretations using four large language models. The overall goal is to evaluate whether humans and LLMs are aligned in their processing of garden-path sentences. Experiments show promising alignment between humans and LLMs in the processing of garden-path sentences.
  arXiv  Detail & Related papers  (2024-05-25T03:36:13Z)
• That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context? [64.38544995251642]
  We study semantic ambiguities that exist in the source (English in this work) itself. We focus on idioms that are open to both literal and figurative interpretations. We find that current MT models consistently translate English idioms literally, even when the context suggests a figurative interpretation.
  arXiv  Detail & Related papers  (2023-10-23T06:38:49Z)
• LMs stand their Ground: Investigating the Effect of Embodiment in Figurative Language Interpretation by Language Models [0.0]
  Figurative language is a challenge for language models since its interpretation deviates from their conventional order and meaning. Yet, humans can easily understand and interpret metaphors as they can be derived from embodied metaphors. This study shows how larger language models perform better at interpreting metaphoric sentences when the action of the metaphorical sentence is more embodied.
  arXiv  Detail & Related papers  (2023-05-05T11:44:12Z)
• Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models [91.3755431537592]
  Representing compositional and non-compositional phrases is critical for language understanding. We first formulate a problem of predicting the LM-internal representations of longer phrases given those of their constituents. While we would expect the predictive accuracy to correlate with human judgments of semantic compositionality, we find this is largely not the case.
  arXiv  Detail & Related papers  (2022-10-07T14:21:30Z)
• Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation [55.52888815590317]
  Unlike literal expressions, idioms' meanings do not directly follow from their parts. NMT models are often unable to translate idioms accurately and over-generate compositional, literal translations. We investigate whether the non-compositionality of idioms is reflected in the mechanics of the dominant NMT model, Transformer.
  arXiv  Detail & Related papers  (2022-05-30T17:59:32Z)
• It's not Rocket Science : Interpreting Figurative Language in Narratives [48.84507467131819]
  We study the interpretation of two non-compositional figurative languages (idioms and similes) Our experiments show that models based solely on pre-trained language models perform substantially worse than humans on these tasks. We additionally propose knowledge-enhanced models, adopting human strategies for interpreting figurative language.
  arXiv  Detail & Related papers  (2021-08-31T21:46:35Z)
• Did they answer? Subjective acts and intents in conversational discourse [48.63528550837949]
  We present the first discourse dataset with multiple and subjective interpretations of English conversation. We show disagreements are nuanced and require a deeper understanding of the different contextual factors.
  arXiv  Detail & Related papers  (2021-04-09T16:34:19Z)
• Interpreting Verbal Metaphors by Paraphrasing [12.750941606061877]
  We show that our paraphrasing method significantly outperforms the state-of-the-art baseline. We also demonstrate that our method can help a machine translation system improve its accuracy in translating English metaphors to 8 target languages.
  arXiv  Detail & Related papers  (2021-04-07T21:00:23Z)
• Are Interpretations Fairly Evaluated? A Definition Driven Pipeline for Post-Hoc Interpretability [54.85658598523915]
  We propose to have a concrete definition of interpretation before we could evaluate faithfulness of an interpretation. We find that although interpretation methods perform differently under a certain evaluation metric, such a difference may not result from interpretation quality or faithfulness.
  arXiv  Detail & Related papers  (2020-09-16T06:38:03Z)
• Metaphoric Paraphrase Generation [58.592750281138265]
  We use crowdsourcing to evaluate our results, as well as developing an automatic metric for evaluating metaphoric paraphrases. We show that while the lexical replacement baseline is capable of producing accurate paraphrases, they often lack metaphoricity. Our metaphor masking model excels in generating metaphoric sentences while performing nearly as well with regard to fluency and paraphrase quality.
  arXiv  Detail & Related papers  (2020-02-28T16:30:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.