BERT(s) to Detect Multiword Expressions

• URL: http://arxiv.org/abs/2208.07832v1
• Date: Tue, 16 Aug 2022 16:32:23 GMT
• Title: BERT(s) to Detect Multiword Expressions
• Authors: Damith Premasiri and Tharindu Ranasinghe
• Abstract summary: Multiword expressions (MWEs) present groups of words in which the meaning of the whole is not derived from the meaning of its parts. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs.
• Score: 9.710464466895521
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multiword expressions (MWEs) present groups of words in which the meaning of the whole is not derived from the meaning of its parts. The task of processing MWEs is crucial in many natural language processing (NLP) applications, including machine translation and terminology extraction. Therefore, detecting MWEs is a popular research theme. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs.We empirically evaluate several transformer models in the dataset for SemEval-2016 Task 10: Detecting Minimal Semantic Units and their Meanings (DiMSUM). We show that transformer models outperform the previous neural models based on long short-term memory (LSTM). The code and pre-trained model will be made freely available to the community.

Related papers

• Interpretable Language Modeling via Induction-head Ngram Models [74.26720927767398]
  We propose Induction-head ngram models (Induction-Gram) to bolster modern ngram models with a hand-engineered "induction head" This induction head uses a custom neural similarity metric to efficiently search the model's input context for potential next-word completions. Experiments show that this simple method significantly improves next-word prediction over baseline interpretable models.
  arXiv  Detail & Related papers  (2024-10-31T12:33:26Z)
• Segment-Based Interactive Machine Translation for Pre-trained Models [2.0871483263418806]
  We explore the use of pre-trained large language models (LLM) in interactive machine translation environments. The system generates perfect translations interactively using the feedback provided by the user at each iteration. We compare the performance of mBART, mT5 and a state-of-the-art (SoTA) machine translation model on a benchmark dataset regarding user effort.
  arXiv  Detail & Related papers  (2024-07-09T16:04:21Z)
• MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining [73.81862342673894]
  Foundation models have reshaped the landscape of Remote Sensing (RS) by enhancing various image interpretation tasks. transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks. We conduct multi-task supervised pretraining on the SAMRS dataset, encompassing semantic segmentation, instance segmentation, and rotated object detection. Our models are finetuned on various RS downstream tasks, such as scene classification, horizontal and rotated object detection, semantic segmentation, and change detection.
  arXiv  Detail & Related papers  (2024-03-20T09:17:22Z)
• Semantics of Multiword Expressions in Transformer-Based Models: A Survey [8.372465442144048]
  Multiword expressions (MWEs) are composed of multiple words and exhibit variable degrees of compositionality. We provide the first in-depth survey of MWE processing with transformer models. We find that they capture MWE semantics inconsistently, as shown by reliance on surface patterns and memorized information.
  arXiv  Detail & Related papers  (2024-01-27T11:51:11Z)
• HilMeMe: A Human-in-the-Loop Machine Translation Evaluation Metric Looking into Multi-Word Expressions [6.85316573653194]
  We describe the design and implementation of a linguistically motivated human-in-the-loop evaluation metric looking into idiomatic and terminological Multi-word Expressions (MWEs) MWEs can be used as one of the main factors to distinguish different MT systems by looking into their capabilities in recognising and translating MWEs in an accurate and meaning equivalent manner.
  arXiv  Detail & Related papers  (2022-11-09T21:15:40Z)
• Exploring Modulated Detection Transformer as a Tool for Action Recognition in Videos [0.0]
  Modulated Detection Transformer (MDETR) is an end-to-end multi-modal understanding model. We show that it is possible to use a multi-modal model to tackle a task that it was not designed for.
  arXiv  Detail & Related papers  (2022-09-21T05:19:39Z)
• Transformer-based Detection of Multiword Expressions in Flower and Plant Names [9.281156301926769]
  Multiword expression (MWE) is a sequence of words which collectively present a meaning which is not derived from its individual words. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs in flower and plant names.
  arXiv  Detail & Related papers  (2022-09-16T15:59:55Z)
• Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation [50.54059385277964]
  We present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CsaNMT) CsaNMT augments each training instance with an adjacency region that could cover adequate variants of literal expression under the same meaning.
  arXiv  Detail & Related papers  (2022-04-14T08:16:28Z)
• An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Tasks [112.1942546460814]
  We report the first exploration of the prompt tuning paradigm for speech processing tasks based on Generative Spoken Language Model (GSLM) Experiment results show that the prompt tuning technique achieves competitive performance in speech classification tasks with fewer trainable parameters than fine-tuning specialized downstream models.
  arXiv  Detail & Related papers  (2022-03-31T03:26:55Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Bayesian Transformer Language Models for Speech Recognition [59.235405107295655]
  State-of-the-art neural language models (LMs) represented by Transformers are highly complex. This paper proposes a full Bayesian learning framework for Transformer LM estimation.
  arXiv  Detail & Related papers  (2021-02-09T10:55:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.