Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification

• URL: http://arxiv.org/abs/2606.07479v1
• Date: Fri, 05 Jun 2026 17:34:07 GMT
• Title: Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification
• Abstract summary: Turkish idiomatic light verb constructions (LVCs) are challenging for multiword expression processing.<n>We frame Turkish LVC detection as a binary classification task (literal meaning vs. idiomatic meaning)<n>We evaluate on a manually created controlled set (N=147) with matched negatives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Turkish idiomatic light verb constructions (LVCs) are challenging for multiword expression processing because they often share the same surface form as fully literal verb-object combinations while functioning as a single, partially idiomatic predicate. We frame Turkish LVC detection as a binary classification task (literal meaning vs. idiomatic meaning) and evaluate on a manually created controlled set (N=147) with matched negatives: out-of-domain random sentences and in-domain literal controls (NLVC), alongside LVC positives. We compare a supervised Turkish encoder baseline (BERTurk with a classifier head) to three instruction-tuned LLMs from different families under zero-shot, one-shot, and few-shot prompting, and analyze how demonstrations shift error profiles. In zero-shot, LLMs perform well on negatives but show very low LVC recall. One-shot prompting sharply improves LVC detection but can induce strong, model-specific biases, leading models to overpredict or underpredict LVCs. A richer few-shot prompt improves calibration and yields robust overall performance for GPT-OSS-20B and Qwen 2.5-14B. Overall, the results highlight substantial prompt sensitivity in Turkish metalinguistic classification: the supervised baseline remains competitive, while prompted LLMs can match or exceed it on LVCs with carefully constructed demonstrations.

Related papers

• Multilingual Cognitive Impairment Detection in the Era of Foundation Models [6.141505279958125]
  We evaluate cognitive impairment (CI) classification from transcripts of speech in English, Slovene, and Korean.<n>We compare zero-shot large language models (LLMs) used as direct classifiers under three input settings.
  arXiv  Detail & Related papers  (2026-04-08T07:22:43Z)
• Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity [6.285055089721361]
  We introduce a principled interpretability framework based on token-level perplexity to test whether models rely on linguistically relevant cues.<n>Our method enables precise, hypothesis-driven analysis without relying on unstable feature-attribution techniques.
  arXiv  Detail & Related papers  (2026-03-31T08:00:55Z)
• From Lemmas to Dependencies: What Signals Drive Light Verbs Classification? [0.0]
  Light verb constructions (LVCs) are a challenging class of verbal multiword expressions, especially in Turkish.<n>This paper asks what signals drive LVC classification by systematically restricting model inputs.
  arXiv  Detail & Related papers  (2026-02-04T01:41:11Z)
• Language-Guided Invariance Probing of Vision-Language Models [2.0789144738685956]
  We introduce Language-Guided Invariance Probing (LGIP) to measure invariance to paraphrases and sensitivity to semantic flips in image-text matching.<n>We show that EVA02-CLIP and large OpenCLIP variants lie on a favorable invariance-sensitivity frontier, combining low paraphrase-induced variance with consistently higher scores for original captions.
  arXiv  Detail & Related papers  (2025-11-17T15:35:49Z)
• Bridging the Gap: In-Context Learning for Modeling Human Disagreement [8.011316959982654]
  Large Language Models (LLMs) have shown strong performance on NLP classification tasks.<n>This study examines whether LLMs can capture multiple perspectives and reflect annotator disagreement in subjective tasks such as hate speech and offensive language detection.
  arXiv  Detail & Related papers  (2025-06-06T14:24:29Z)
• Aligning Language Models with Demonstrated Feedback [58.834937450242975]
  Demonstration ITerated Task Optimization (DITTO) directly aligns language model outputs to a user's demonstrated behaviors.<n>We evaluate DITTO's ability to learn fine-grained style and task alignment across domains such as news articles, emails, and blog posts.
  arXiv  Detail & Related papers  (2024-06-02T23:13:56Z)
• Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
  We propose a new evaluation method, SQC-Score. Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score. Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
  arXiv  Detail & Related papers  (2024-04-04T15:36:53Z)
• Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models [76.410400238974]
  We propose TTA with feedback to rectify the model output and prevent the model from becoming blindly confident. A CLIP model is adopted as the reward model during TTA and provides feedback for the VLM. The proposed textitreinforcement learning with CLIP feedback(RLCF) framework is highly flexible and universal.
  arXiv  Detail & Related papers  (2023-05-29T11:03:59Z)
• Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds [59.71218039095155]
  We evaluate language understanding capacities on simple inference tasks that most humans find trivial. We target (i) grammatically-specified entailments, (ii) premises with evidential adverbs of uncertainty, and (iii) monotonicity entailments. The models exhibit moderate to low performance on these evaluation sets.
  arXiv  Detail & Related papers  (2023-05-24T06:41:09Z)
• Understanding Emergent In-Context Learning from a Kernel Regression Perspective [55.95455089638838]
  Large language models (LLMs) have initiated a paradigm shift in transfer learning.<n>This paper proposes a kernel-regression perspective of understanding LLMs' ICL bahaviors when faced with in-context examples.<n>We find that during ICL, the attention and hidden features in LLMs match the behaviors of a kernel regression.
  arXiv  Detail & Related papers  (2023-05-22T06:45:02Z)
• Alleviating Over-smoothing for Unsupervised Sentence Representation [96.19497378628594]
  We present a Simple method named Self-Contrastive Learning (SSCL) to alleviate this issue. Our proposed method is quite simple and can be easily extended to various state-of-the-art models for performance boosting.
  arXiv  Detail & Related papers  (2023-05-09T11:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.