NCL-UoR at SemEval-2026 Task 5: Embedding-Based Methods, Fine-Tuning, and LLMs for Word Sense Plausibility Rating

• URL: http://arxiv.org/abs/2603.08256v1
• Date: Mon, 09 Mar 2026 11:25:41 GMT
• Title: NCL-UoR at SemEval-2026 Task 5: Embedding-Based Methods, Fine-Tuning, and LLMs for Word Sense Plausibility Rating
• Abstract summary: Word sense plausibility rating requires predicting the human-perceived plausibility of a given word on a 1--5 scale.<n>This paper systematically compares three approaches to predicting word sense plausibility.<n>The best-performing system employs a structured prompting strategy that decomposes evaluation into narrative components.
• Score: 5.91164076667492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Word sense plausibility rating requires predicting the human-perceived plausibility of a given word sense on a 1--5 scale in the context of short narrative stories containing ambiguous homonyms. This paper systematically compares three approaches: (1) embedding-based methods pairing sentence embeddings with standard regressors, (2) transformer fine-tuning with parameter-efficient adaptation, and (3) large language model (LLM) prompting with structured reasoning and explicit decision rules. The best-performing system employs a structured prompting strategy that decomposes evaluation into narrative components (precontext, target sentence, ending) and applies explicit decision rules for rating calibration. The analysis reveals that structured prompting with decision rules substantially outperforms both fine-tuned models and embedding-based approaches, and that prompt design matters more than model scale for this task. The code is publicly available at https://github.com/tongwu17/SemEval-2026-Task5.

Related papers

• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News [1.4465033892011254]
  We present two approaches for classifying sentences from German-language climate news articles as threat-oriented, solution-oriented, both, or neither.<n>The first approach uses few-shot prompting with an open-weights large language model (Llama 4 Maverick), employing chain-of-thought reasoning and structured output with confidence scoring.<n>The second approach fine-tunes a German BERT model (deepset/gbert-large) for sentence-pair classification, where the preceding sentence provides contextual information for the target sentence.
  arXiv  Detail & Related papers  (2026-06-25T00:48:40Z)
• ClaHF: A Human Feedback-inspired Reinforcement Learning Framework for Improving Classification Tasks [11.138973570127206]
  We propose ClaHF, a human feedback-inspired reinforcement learning framework for text classification.<n>ClaHF integrates preference modeling and RL optimization into the classification pipeline without requiring additional human annotations.<n>Results demonstrate that ClaHF consistently improves both classification performance and confidence calibration across diverse language models.
  arXiv  Detail & Related papers  (2026-05-17T14:00:01Z)
• COGNAC at SemEval-2026 Task 5: LLM Ensembles for Human-Level Word Sense Plausibility Rating in Challenging Narratives [0.0]
  We describe our system for rating the plausibility of given word senses of homonyms in short stories on a 5-point Likert scale.<n>System is evaluated by the unweighted average of accuracy (within one standard deviation of mean human judgments) and Spearman Rank Correlation.
  arXiv  Detail & Related papers  (2026-03-16T20:42:02Z)
• VERGE: Formal Refinement and Guidance Engine for Verifiable LLM Reasoning [4.3414302048068745]
  We present a neurosymbolic framework that combines Large Language Models with SMT solvers to produce verification-guided answers.<n>We introduce three key innovations: (1) multi-model consensus via formal semantic equivalence checking, (2) semantic routing that directs different claim types to appropriate verification strategies, and (3) precise logical error localization via Minimal Correction Subsets.<n>With the GPT-OSS-120B model, VERGE demonstrates an average performance uplift of 18.7% at convergence across a set of reasoning benchmarks compared to single-pass approaches.
  arXiv  Detail & Related papers  (2026-01-27T20:59:11Z)
• DiSC-AMC: Token- and Parameter-Efficient Discretized Statistics In-Context Automatic Modulation Classification [19.190236060870184]
  Large Language Models (LLMs) can perform Automatic Modulation Classification (AMC) in an open-set manner without fine-tuning.<n>We present Discretized Statistics in-Context Automatic Modulation Classification (DiSC-AMC)<n>DiSC-AMC is a token- and parameter-efficient variant that discretizes higher-order statistics and cumulants into compact symbolic tokens.
  arXiv  Detail & Related papers  (2025-09-30T22:20:57Z)
• Vec2Summ: Text Summarization via Probabilistic Sentence Embeddings [2.2029818765681086]
  Vec2Summ represents a document collection using a single mean vector in the semantic embedding space.<n>We reconstruct fluent summaries using a generative language model.<n>Vec2Summ produces coherent summaries for topically focused, order-invariant corpora.
  arXiv  Detail & Related papers  (2025-08-09T15:31:02Z)
• Syntactic Control of Language Models by Posterior Inference [53.823006836309695]
  Controlling the syntactic structure of text generated by language models is valuable for applications requiring clarity, stylistic consistency, or interpretability.<n>We argue that sampling algorithms based on the posterior inference can effectively enforce a target constituency structure during generation.<n>Our approach combines sequential Monte Carlo, which estimates the posterior distribution by sampling from a proposal distribution, with a syntactic tagger that ensures that each generated token aligns with the desired syntactic structure.
  arXiv  Detail & Related papers  (2025-06-08T14:01:34Z)
• Analysing Zero-Shot Readability-Controlled Sentence Simplification [54.09069745799918]
  We investigate how different types of contextual information affect a model's ability to generate sentences with the desired readability.<n>Results show that all tested models struggle to simplify sentences due to models' limitations and characteristics of the source sentences.<n>Our experiments also highlight the need for better automatic evaluation metrics tailored to RCTS.
  arXiv  Detail & Related papers  (2024-09-30T12:36:25Z)
• Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
  We propose a new evaluation method, SQC-Score. Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score. Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
  arXiv  Detail & Related papers  (2024-04-04T15:36:53Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Improving the Generalization Ability in Essay Coherence Evaluation through Monotonic Constraints [22.311428543432605]
  Coherence is a crucial aspect of evaluating text readability and can be assessed through two primary factors. We propose a coherence scoring model consisting of a regression model with two feature extractors. The model achieved third place in track 1 of NLPCC 2023 shared task 7.
  arXiv  Detail & Related papers  (2023-07-25T08:26:46Z)
• Evaluating Text Coherence at Sentence and Paragraph Levels [17.99797111176988]
  We investigate the adaptation of existing sentence ordering methods to a paragraph ordering task. We also compare the learnability and robustness of existing models by artificially creating mini datasets and noisy datasets. We conclude that the recurrent graph neural network-based model is an optimal choice for coherence modeling.
  arXiv  Detail & Related papers  (2020-06-05T03:31:49Z)
• Pseudo-Convolutional Policy Gradient for Sequence-to-Sequence Lip-Reading [96.48553941812366]
  Lip-reading aims to infer the speech content from the lip movement sequence. Traditional learning process of seq2seq models suffers from two problems. We propose a novel pseudo-convolutional policy gradient (PCPG) based method to address these two problems.
  arXiv  Detail & Related papers  (2020-03-09T09:12:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.