Auto-tagging of Short Conversational Sentences using Natural Language Processing Methods

• URL: http://arxiv.org/abs/2106.04959v1
• Date: Wed, 9 Jun 2021 10:14:05 GMT
• Title: Auto-tagging of Short Conversational Sentences using Natural Language Processing Methods
• Authors: \c{S}\"ukr\"u Ozan, D. Emre Ta\c{s}ar
• Abstract summary: We manually tagged approximately 14 thousand visitor inputs into ten basic categories. We considered three different state-of-the-art models and reported their auto-tagging capabilities. Implementation of the models used in these experiments can be cloned from our GitHub repository and tested for similar auto-tagging problems without much effort.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this study, we aim to find a method to auto-tag sentences specific to a domain. Our training data comprises short conversational sentences extracted from chat conversations between company's customer representatives and web site visitors. We manually tagged approximately 14 thousand visitor inputs into ten basic categories, which will later be used in a transformer-based language model with attention mechanisms for the ultimate goal of developing a chatbot application that can produce meaningful dialogue. We considered three different state-of-the-art models and reported their auto-tagging capabilities. We achieved the best performance with the bidirectional encoder representation from transformers (BERT) model. Implementation of the models used in these experiments can be cloned from our GitHub repository and tested for similar auto-tagging problems without much effort.

Related papers

• Representing Rule-based Chatbots with Transformers [35.30128900987116]
  We build on prior work by constructing a Transformer that implements the ELIZA program. ELIZA illustrates some of the distinctive challenges of the conversational setting. We train Transformers on a dataset of synthetically generated ELIZA conversations and investigate the mechanisms the models learn.
  arXiv  Detail & Related papers  (2024-07-15T17:45:53Z)
• Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts [11.067252960486272]
  We present a simple yet general method to simulate real-time interactive conversations using pretrained language models. We demonstrate the promise of this method with two case studies: instant messenger dialogues and spoken conversations.
  arXiv  Detail & Related papers  (2024-05-21T21:14:31Z)
• Computational Argumentation-based Chatbots: a Survey [0.4024850952459757]
  The present survey sifts through the literature to review papers concerning this kind of argumentation-based bot. It draws conclusions about the drawbacks and benefits of this approach. It also envisaging possible future development and integration with the Transformer-based architecture and state-of-the-art Large Language models.
  arXiv  Detail & Related papers  (2024-01-07T11:20:42Z)
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control [140.48218261864153]
  We study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control. Our approach leads to performant robotic policies and enables RT-2 to obtain a range of emergent capabilities from Internet-scale training.
  arXiv  Detail & Related papers  (2023-07-28T21:18:02Z)
• Stateful Memory-Augmented Transformers for Efficient Dialogue Modeling [69.31802246621963]
  We propose a novel memory-augmented transformer that is compatible with existing pre-trained encoder-decoder models. By incorporating a separate memory module alongside the pre-trained transformer, the model can effectively interchange information between the memory states and the current input context.
  arXiv  Detail & Related papers  (2022-09-15T22:37:22Z)
• GODEL: Large-Scale Pre-Training for Goal-Directed Dialog [119.1397031992088]
  We introduce GODEL, a large pre-trained language model for dialog. We show that GODEL outperforms state-of-the-art pre-trained dialog models in few-shot fine-tuning setups. A novel feature of our evaluation methodology is the introduction of a notion of utility that assesses the usefulness of responses.
  arXiv  Detail & Related papers  (2022-06-22T18:19:32Z)
• Vector Representations of Idioms in Conversational Systems [1.6507910904669727]
  We utilize the Potentialatic Expression (PIE)-English idioms corpus for the two tasks that we investigate. We achieve state-of-the-art (SoTA) result of 98% macro F1 score on the classification task by using the SoTA T5 model. The results show that the model trained on the idiom corpus generates more fitting responses to prompts containing idioms 71.9% of the time.
  arXiv  Detail & Related papers  (2022-05-07T14:50:05Z)
• DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation [80.45816053153722]
  DialogVED introduces continuous latent variables into the enhanced encoder-decoder pre-training framework to increase the relevance and diversity of responses. We conduct experiments on PersonaChat, DailyDialog, and DSTC7-AVSD benchmarks for response generation.
  arXiv  Detail & Related papers  (2022-04-27T16:18:15Z)
• Sentence Bottleneck Autoencoders from Transformer Language Models [53.350633961266375]
  We build a sentence-level autoencoder from a pretrained, frozen transformer language model. We adapt the masked language modeling objective as a generative, denoising one, while only training a sentence bottleneck and a single-layer modified transformer decoder. We demonstrate that the sentence representations discovered by our model achieve better quality than previous methods that extract representations from pretrained transformers on text similarity tasks, style transfer, and single-sentence classification tasks in the GLUE benchmark, while using fewer parameters than large pretrained models.
  arXiv  Detail & Related papers  (2021-08-31T19:39:55Z)
• CloneBot: Personalized Dialogue-Response Predictions [0.0]
  The project task was to create a model that, given a speaker ID, chat history, and an utterance query, can predict the response utterance in a conversation. The model is personalized for each speaker. This task can be a useful tool for building speech bots that talk in a human-like manner in a live conversation.
  arXiv  Detail & Related papers  (2021-03-31T01:15:37Z)
• The Adapter-Bot: All-In-One Controllable Conversational Model [66.48164003532484]
  We propose a dialogue model that uses a fixed backbone model such as DialGPT and triggers on-demand dialogue skills via different adapters. Depending on the skills, the model is able to process multiple knowledge types, such as text, tables, and emphatic responses. We evaluate our model using automatic evaluation by comparing it with existing state-of-the-art conversational models.
  arXiv  Detail & Related papers  (2020-08-28T10:59:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.