Odor Descriptor Understanding through Prompting

• URL: http://arxiv.org/abs/2205.03719v1
• Date: Sat, 7 May 2022 20:44:22 GMT
• Title: Odor Descriptor Understanding through Prompting
• Authors: Laura Sisson
• Abstract summary: We present two methods to generate embeddings for odor words that are more closely aligned with their olfactory meanings. These generated embeddings outperform the previous state-of-the-art and contemporary fine-tuning/prompting methods on a pre-existing zero-shot odor-specific NLP benchmark.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embeddings from contemporary natural language processing (NLP) models are commonly used as numerical representations for words or sentences. However, odor descriptor words, like "leather" or "fruity", vary significantly between their commonplace usage and their olfactory usage, as a result traditional methods for generating these embeddings do not suffice. In this paper, we present two methods to generate embeddings for odor words that are more closely aligned with their olfactory meanings when compared to off-the-shelf embeddings. These generated embeddings outperform the previous state-of-the-art and contemporary fine-tuning/prompting methods on a pre-existing zero-shot odor-specific NLP benchmark.

Related papers

• Fine-tuning Language Models for Factuality [96.5203774943198]
  Large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations' In this work, we fine-tune language models to be more factual, without human labeling.
  arXiv  Detail & Related papers  (2023-11-14T18:59:15Z)
• CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classification [57.62886091828512]
  We propose a brand-new prefix-tuning method, Counterfactual Contrastive Prefix-tuning (CCPrefix) for many-class classification. Basically, an instance-dependent soft prefix, derived from fact-counterfactual pairs in the label space, is leveraged to complement the language verbalizers in many-class classification.
  arXiv  Detail & Related papers  (2022-11-11T03:45:59Z)
• From meaning to perception -- exploring the space between word and odor perception embeddings [0.30458514384586394]
  We propose the use of the Word2vec algorithm in order to obtain odor perception embeddings (or smell embeddings) The meaningfulness of these embeddings suggests that aesthetics might provide enough constraints for using algorithms motivated by distributional semantics. We have also employed the embeddings in an attempt to understand the aesthetic nature of perfumes, based on the difference between real and randomly generated perfumes.
  arXiv  Detail & Related papers  (2022-03-19T10:44:20Z)
• Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost [5.672132510411465]
  State-of-the-art NLP systems represent inputs with word embeddings, but these are brittle when faced with Out-of-Vocabulary words. We follow the principle of mimick-like models to generate vectors for unseen words, by learning the behavior of pre-trained embeddings using only the surface form of words. We present a simple contrastive learning framework, LOVE, which extends the word representation of an existing pre-trained language model (such as BERT) and makes it robust to OOV with few additional parameters.
  arXiv  Detail & Related papers  (2022-03-15T13:11:07Z)
• Lacking the embedding of a word? Look it up into a traditional dictionary [0.2624902795082451]
  We propose to use definitions retrieved in traditional dictionaries to produce word embeddings for rare words. DefiNNet and DefBERT significantly outperform state-of-the-art as well as baseline methods for producing embeddings of unknown words.
  arXiv  Detail & Related papers  (2021-09-24T06:27:58Z)
• Prompting Contrastive Explanations for Commonsense Reasoning Tasks [74.7346558082693]
  Large pretrained language models (PLMs) can achieve near-human performance on commonsense reasoning tasks. We show how to use these same models to generate human-interpretable evidence.
  arXiv  Detail & Related papers  (2021-06-12T17:06:13Z)
• Obtaining Better Static Word Embeddings Using Contextual Embedding Models [53.86080627007695]
  Our proposed distillation method is a simple extension of CBOW-based training. As a side-effect, our approach also allows a fair comparison of both contextual and static embeddings.
  arXiv  Detail & Related papers  (2021-06-08T12:59:32Z)
• Unnatural Language Inference [48.45003475966808]
  We find that state-of-the-art NLI models, such as RoBERTa and BART, are invariant to, and sometimes even perform better on, examples with randomly reordered words. Our findings call into question the idea that our natural language understanding models, and the tasks used for measuring their progress, genuinely require a human-like understanding of syntax.
  arXiv  Detail & Related papers  (2020-12-30T20:40:48Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• Deconstructing word embedding algorithms [17.797952730495453]
  We propose a retrospective on some of the most well-known word embedding algorithms. In this work, we deconstruct Word2vec, GloVe, and others, into a common form, unveiling some of the common conditions that seem to be required for making performant word embeddings.
  arXiv  Detail & Related papers  (2020-11-12T14:23:35Z)
• GiBERT: Introducing Linguistic Knowledge into BERT through a Lightweight Gated Injection Method [29.352569563032056]
  We propose a novel method to explicitly inject linguistic knowledge in the form of word embeddings into a pre-trained BERT. Our performance improvements on multiple semantic similarity datasets when injecting dependency-based and counter-fitted embeddings indicate that such information is beneficial and currently missing from the original model.
  arXiv  Detail & Related papers  (2020-10-23T17:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.