Generative Pre-trained Transformer for Vietnamese Community-based COVID-19 Question Answering

• URL: http://arxiv.org/abs/2310.14602v2
• Date: Tue, 31 Oct 2023 09:15:51 GMT
• Title: Generative Pre-trained Transformer for Vietnamese Community-based COVID-19 Question Answering
• Authors: Tam Minh Vo and Khiem Vinh Tran
• Abstract summary: Generative Pre-trained Transformer (GPT) has been effectively employed as a decoder within state-of-the-art (SOTA) question answering systems. This paper presents an implementation of GPT-2 for community-based question answering specifically focused on COVID-19 related queries in Vietnamese.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent studies have provided empirical evidence of the wide-ranging potential of Generative Pre-trained Transformer (GPT), a pretrained language model, in the field of natural language processing. GPT has been effectively employed as a decoder within state-of-the-art (SOTA) question answering systems, yielding exceptional performance across various tasks. However, the current research landscape concerning GPT's application in Vietnamese remains limited. This paper aims to address this gap by presenting an implementation of GPT-2 for community-based question answering specifically focused on COVID-19 related queries in Vietnamese. We introduce a novel approach by conducting a comparative analysis of different Transformers vs SOTA models in the community-based COVID-19 question answering dataset. The experimental findings demonstrate that the GPT-2 models exhibit highly promising outcomes, outperforming other SOTA models as well as previous community-based COVID-19 question answering models developed for Vietnamese.

Related papers

• On Training Data Influence of GPT Models [37.53037752668756]
  GPTfluence is a novel approach to assess the impact of training examples on the training dynamics of GPT models. Our approach traces the influence of individual training instances on performance trajectories, such as loss and other key metrics, on targeted test points.
  arXiv  Detail & Related papers  (2024-04-11T15:27:56Z)
• Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases [98.35348038111508]
  This paper presents an in-depth comparative study of two pioneering models: Google's Gemini and OpenAI's GPT-4V(ision) The core of our analysis delves into the distinct visual comprehension abilities of each model. Our findings illuminate the unique strengths and niches of both models.
  arXiv  Detail & Related papers  (2023-12-22T18:59:58Z)
• BARTPhoBEiT: Pre-trained Sequence-to-Sequence and Image Transformers Models for Vietnamese Visual Question Answering [3.0938904602244355]
  Visual Question Answering (VQA) is an intricate and demanding task that integrates natural language processing (NLP) and computer vision (CV) We introduce a transformer-based Vietnamese model named BARTPhoBEiT. This model includes pre-trained Sequence-to-Sequence and bidirectional encoder representation from Image Transformers in Vietnamese and evaluates Vietnamese VQA datasets.
  arXiv  Detail & Related papers  (2023-07-28T06:23:32Z)
• Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation [114.80518907146792]
  We investigate the potential of utilizing large-scale language models, such as GPT-k, to improve the prompt editing process for text-to-image generation. We compare the common edits made by humans and GPT-k, evaluate the performance of GPT-k in prompting T2I, and examine factors that may influence this process.
  arXiv  Detail & Related papers  (2023-05-18T21:53:58Z)
• Multi-Aspect Explainable Inductive Relation Prediction by Sentence Transformer [60.75757851637566]
  We introduce the concepts of relation path coverage and relation path confidence to filter out unreliable paths prior to model training to elevate the model performance. We propose Knowledge Reasoning Sentence Transformer (KRST) to predict inductive relations in knowledge graphs.
  arXiv  Detail & Related papers  (2023-01-04T15:33:49Z)
• BJTU-WeChat's Systems for the WMT22 Chat Translation Task [66.81525961469494]
  This paper introduces the joint submission of the Beijing Jiaotong University and WeChat AI to the WMT'22 chat translation task for English-German. Based on the Transformer, we apply several effective variants. Our systems achieve 0.810 and 0.946 COMET scores.
  arXiv  Detail & Related papers  (2022-11-28T02:35:04Z)
• UIT-ViCoV19QA: A Dataset for COVID-19 Community-based Question Answering on Vietnamese Language [0.0]
  We present the first Vietnamese community-based question answering dataset for developing question answering systems for COVID-19 called UIT-ViCoV19QA. The dataset comprises 4,500 question-answer pairs collected from trusted medical sources, with at least one answer and at most four unique paraphrased answers per question.
  arXiv  Detail & Related papers  (2022-09-14T14:24:23Z)
• Elaboration-Generating Commonsense Question Answering at Scale [77.96137534751445]
  In question answering requiring common sense, language models (e.g., GPT-3) have been used to generate text expressing background knowledge. We finetune smaller language models to generate useful intermediate context, referred to here as elaborations. Our framework alternates between updating two language models -- an elaboration generator and an answer predictor -- allowing each to influence the other.
  arXiv  Detail & Related papers  (2022-09-02T18:32:09Z)
• VieSum: How Robust Are Transformer-based Models on Vietnamese Summarization? [1.1379578593538398]
  We investigate the robustness of transformer-based encoder-decoder architectures for Vietnamese abstractive summarization. We validate the performance of the methods on two Vietnamese datasets.
  arXiv  Detail & Related papers  (2021-10-08T17:10:31Z)
• COVID-19 Named Entity Recognition for Vietnamese [6.17059264011429]
  We present the first manually-annotated COVID-19 domain-specific dataset for Vietnamese. Our dataset is annotated for the named entity recognition task with newly-defined entity types. Our dataset also contains the largest number of entities compared to existing Vietnamese NER datasets.
  arXiv  Detail & Related papers  (2021-04-08T16:35:34Z)
• Learning Invariant Representations across Domains and Tasks [81.30046935430791]
  We propose a novel Task Adaptation Network (TAN) to solve this unsupervised task transfer problem. In addition to learning transferable features via domain-adversarial training, we propose a novel task semantic adaptor that uses the learning-to-learn strategy to adapt the task semantics. TAN significantly increases the recall and F1 score by 5.0% and 7.8% compared to recently strong baselines.
  arXiv  Detail & Related papers  (2021-03-03T11:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.