Transformer-based Detection of Multiword Expressions in Flower and Plant Names

• URL: http://arxiv.org/abs/2209.08016v1
• Date: Fri, 16 Sep 2022 15:59:55 GMT
• Title: Transformer-based Detection of Multiword Expressions in Flower and Plant Names
• Authors: Damith Premasiri, Amal Haddad Haddad, Tharindu Ranasinghe, and Ruslan Mitkov
• Abstract summary: Multiword expression (MWE) is a sequence of words which collectively present a meaning which is not derived from its individual words. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs in flower and plant names.
• Score: 9.281156301926769
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multiword expression (MWE) is a sequence of words which collectively present a meaning which is not derived from its individual words. The task of processing MWEs is crucial in many natural language processing (NLP) applications, including machine translation and terminology extraction. Therefore, detecting MWEs in different domains is an important research topic. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs in flower and plant names. We evaluate different transformer models on a dataset created from Encyclopedia of Plants and Flower. We empirically show that transformer models outperform the previous neural models based on long short-term memory (LSTM).

Related papers

• Semantics of Multiword Expressions in Transformer-Based Models: A Survey [8.372465442144048]
  Multiword expressions (MWEs) are composed of multiple words and exhibit variable degrees of compositionality. We provide the first in-depth survey of MWE processing with transformer models. We find that they capture MWE semantics inconsistently, as shown by reliance on surface patterns and memorized information.
  arXiv  Detail & Related papers  (2024-01-27T11:51:11Z)
• Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation [52.270712965271656]
  We propose a new model of contextual word representation, not from a neural perspective, but from a purely syntactic and probabilistic perspective. We find that the graph of our model resembles transformers, with correspondences between dependencies and self-attention. Experiments show that our model performs competitively to transformers on small to medium sized datasets.
  arXiv  Detail & Related papers  (2023-11-26T06:56:02Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• Deep Learning Methods for Extracting Metaphorical Names of Flowers and Plants [10.138996506132344]
  We show that discriminative models perform better than GPT-3.5 model with our best performer reporting 92.2349% F1 score in metaphoric flower and plant names identification task.
  arXiv  Detail & Related papers  (2023-05-18T09:22:29Z)
• Detecting Idiomatic Multiword Expressions in Clinical Terminology using Definition-Based Representation Learning [12.30055843580139]
  We develop an effective tool for scoring the idiomaticity of biomedical MWEs based on the degree of similarity between the semantic representations of those MWEs and a weighted average of the representation of their constituents. Our results show that the BioLORD model has a strong ability to identify idiomatic MWEs, not replicated in other models.
  arXiv  Detail & Related papers  (2023-05-11T13:42:58Z)
• Mapping of attention mechanisms to a generalized Potts model [50.91742043564049]
  We show that training a neural network is exactly equivalent to solving the inverse Potts problem by the so-called pseudo-likelihood method. We also compute the generalization error of self-attention in a model scenario analytically using the replica method.
  arXiv  Detail & Related papers  (2023-04-14T16:32:56Z)
• BERT(s) to Detect Multiword Expressions [9.710464466895521]
  Multiword expressions (MWEs) present groups of words in which the meaning of the whole is not derived from the meaning of its parts. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs.
  arXiv  Detail & Related papers  (2022-08-16T16:32:23Z)
• A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation [131.33610549540043]
  We propose a novel graph-based multi-modal fusion encoder for NMT. We first represent the input sentence and image using a unified multi-modal graph. We then stack multiple graph-based multi-modal fusion layers that iteratively perform semantic interactions to learn node representations.
  arXiv  Detail & Related papers  (2020-07-17T04:06:09Z)
• DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator [61.70748716353692]
  Audio Visual Scene-aware Dialog (AVSD) is the task of generating a response for a question with a given scene, video, audio, and the history of previous turns in the dialog. Existing systems for this task employ the transformers or recurrent neural network-based architecture with the encoder-decoder framework. We propose a Multimodal Semantic Transformer Network. It employs a transformer-based architecture with an attention-based word embedding layer that generates words by querying word embeddings.
  arXiv  Detail & Related papers  (2020-04-01T07:10:08Z)
• Variational Transformers for Diverse Response Generation [71.53159402053392]
  Variational Transformer (VT) is a variational self-attentive feed-forward sequence model. VT combines the parallelizability and global receptive field computation of the Transformer with the variational nature of the CVAE. We explore two types of VT: 1) modeling the discourse-level diversity with a global latent variable; and 2) augmenting the Transformer decoder with a sequence of finegrained latent variables.
  arXiv  Detail & Related papers  (2020-03-28T07:48:02Z)
• Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation [8.87546236839959]
  We propose a morphological word segmentation method on the source-side for Neural machine translation (NMT) It incorporates morphology knowledge to preserve the linguistic and semantic information in the word structure while reducing the vocabulary size at training time. It can be utilized as a preprocessing tool to segment the words in agglutinative languages for other natural language processing (NLP) tasks.
  arXiv  Detail & Related papers  (2020-01-02T10:05:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.