Chatbot Interaction with Artificial Intelligence: Human Data Augmentation with T5 and Language Transformer Ensemble for Text Classification

• URL: http://arxiv.org/abs/2010.05990v2
• Date: Thu, 22 Oct 2020 14:33:08 GMT
• Title: Chatbot Interaction with Artificial Intelligence: Human Data Augmentation with T5 and Language Transformer Ensemble for Text Classification
• Authors: Jordan J. Bird, Anik\'o Ek\'art, Diego R. Faria
• Abstract summary: We present the Interaction with Artificial Intelligence (CI-AI) framework as an approach to the training of deep learning chatbots for task classification. The intelligent system augments human-sourced data via artificial paraphrasing in order to generate a large set of training data. We find that all models are improved when training data is augmented by the T5 model.
• Score: 2.492300648514128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we present the Chatbot Interaction with Artificial Intelligence (CI-AI) framework as an approach to the training of deep learning chatbots for task classification. The intelligent system augments human-sourced data via artificial paraphrasing in order to generate a large set of training data for further classical, attention, and language transformation-based learning approaches for Natural Language Processing. Human beings are asked to paraphrase commands and questions for task identification for further execution of a machine. The commands and questions are split into training and validation sets. A total of 483 responses were recorded. Secondly, the training set is paraphrased by the T5 model in order to augment it with further data. Seven state-of-the-art transformer-based text classification algorithms (BERT, DistilBERT, RoBERTa, DistilRoBERTa, XLM, XLM-RoBERTa, and XLNet) are benchmarked for both sets after fine-tuning on the training data for two epochs. We find that all models are improved when training data is augmented by the T5 model, with an average increase of classification accuracy by 4.01%. The best result was the RoBERTa model trained on T5 augmented data which achieved 98.96% classification accuracy. Finally, we found that an ensemble of the five best-performing transformer models via Logistic Regression of output label predictions led to an accuracy of 99.59% on the dataset of human responses. A highly-performing model allows the intelligent system to interpret human commands at the social-interaction level through a chatbot-like interface (e.g. "Robot, can we have a conversation?") and allows for better accessibility to AI by non-technical users.

Related papers

• Distinguishing Chatbot from Human [1.1249583407496218]
  We develop a new dataset consisting of more than 750,000 human-written paragraphs. Based on this dataset, we apply Machine Learning (ML) techniques to determine the origin of text. Our proposed solutions offer high classification accuracy and serve as useful tools for textual analysis.
  arXiv  Detail & Related papers  (2024-08-03T13:18:04Z)
• A Transformer-based Approach for Augmenting Software Engineering Chatbots Datasets [4.311626046942916]
  We present an automated transformer-based approach to augment software engineering datasets. We evaluate the impact of using the augmentation approach on the Rasa NLU's performance using three software engineering datasets.
  arXiv  Detail & Related papers  (2024-07-16T17:48:44Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)
• Investigating Pre-trained Language Models on Cross-Domain Datasets, a Step Closer to General AI [0.8889304968879164]
  We investigate the ability of pre-trained language models to generalize to different non-language tasks. The four pre-trained models that we used, T5, BART, BERT, and GPT-2 achieve outstanding results.
  arXiv  Detail & Related papers  (2023-06-21T11:55:17Z)
• Generate labeled training data using Prompt Programming and GPT-3. An example of Big Five Personality Classification [0.0]
  We generate 25000 conversations labeled with Big Five Personality traits using prompt programming at GPT-3. Then we train Big Five classification models with these data and evaluate them with 2500 data from generated dialogues and real conversational datasets labeled in Big Five by human annotators.
  arXiv  Detail & Related papers  (2023-03-22T03:12:40Z)
• RT-1: Robotics Transformer for Real-World Control at Scale [98.09428483862165]
  We present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks.
  arXiv  Detail & Related papers  (2022-12-13T18:55:15Z)
• HaT5: Hate Language Identification using Text-to-Text Transfer Transformer [1.2532400738980594]
  We investigate the performance of a state-of-the art (SoTA) architecture T5 across 5 different tasks from 2 relatively diverse datasets. To improve performance, we augment the training data by using an autoregressive model. It reveals the difficulties of poor data annotation by using a small set of examples.
  arXiv  Detail & Related papers  (2022-02-11T15:21:27Z)
• Logical Reasoning for Task Oriented Dialogue Systems [57.440956636333325]
  We propose a novel method to fine-tune transformer models such as Roberta and T5 to reason over a set of facts in a given dialogue context. Our method includes a synthetic data generation mechanism which helps the model learn logical relations. We show that the transformer based model can perform logical reasoning to answer questions when the dialogue context contains all the required information.
  arXiv  Detail & Related papers  (2022-02-08T21:46:27Z)
• Few-shot learning through contextual data augmentation [74.20290390065475]
  Machine translation models need to adapt to new data to maintain their performance over time. We show that adaptation on the scale of one to five examples is possible. Our model reports better accuracy scores than a reference system trained with on average 313 parallel examples.
  arXiv  Detail & Related papers  (2021-03-31T09:05:43Z)
• Investigation of Sentiment Controllable Chatbot [50.34061353512263]
  In this paper, we investigate four models to scale or adjust the sentiment of the response. The models are a persona-based model, reinforcement learning, a plug and play model, and CycleGAN. We develop machine-evaluated metrics to estimate whether the responses are reasonable given the input.
  arXiv  Detail & Related papers  (2020-07-11T16:04:30Z)
• Learning Predictive Models From Observation and Interaction [137.77887825854768]
  Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works. However, learning a model that captures the dynamics of complex skills represents a major challenge. We propose a method to augment the training set with observational data of other agents, such as humans.
  arXiv  Detail & Related papers  (2019-12-30T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.