Investigating African-American Vernacular English in Transformer-Based Text Generation

• URL: http://arxiv.org/abs/2010.02510v2
• Date: Thu, 29 Oct 2020 04:00:46 GMT
• Title: Investigating African-American Vernacular English in Transformer-Based Text Generation
• Authors: Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, William Yang Wang
• Abstract summary: Social media has encouraged the written use of African American Vernacular English (AAVE) We investigate the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs. We find that while AAVE text results in more classifications of negative sentiment than SAE, the use of GPT-2 generally increases occurrences of positive sentiment for both.
• Score: 55.53547556060537
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts. However, NLP models have historically been developed using dominant English varieties, such as Standard American English (SAE), due to text corpora availability. We investigate the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs, thereby isolating syntactic structure and AAVE- or SAE-specific language for each pair. We evaluate each sample and its GPT-2 generated text with pretrained sentiment classifiers and find that while AAVE text results in more classifications of negative sentiment than SAE, the use of GPT-2 generally increases occurrences of positive sentiment for both. Additionally, we conduct human evaluation of AAVE and SAE text generated with GPT-2 to compare contextual rigor and overall quality.

Related papers

• Fusion approaches for emotion recognition from speech using acoustic and text-based features [15.186937600119897]
  We study different approaches for classifying emotions from speech using acoustic and text-based features. We compare strategies to combine the audio and text modalities, evaluating them on IEMOCAP and MSP-PODCAST datasets. For IEMOCAP, we show the large effect that the criteria used to define the cross-validation folds have on results.
  arXiv  Detail & Related papers  (2024-03-27T14:40:25Z)
• GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation [93.55550787058012]
  This paper presents an automatic, versatile, and human-aligned evaluation metric for text-to-3D generative models. To this end, we first develop a prompt generator using GPT-4V to generate evaluating prompts. We then design a method instructing GPT-4V to compare two 3D assets according to user-defined criteria.
  arXiv  Detail & Related papers  (2024-01-08T18:52:09Z)
• BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generation [60.77990074569754]
  We present a computation-efficient framework that steers a frozen Pre-Trained Language Model towards more commonsensical generation. Specifically, we first construct a reference-free evaluator that assigns a sentence with a commonsensical score. We then use the scorer as the oracle for commonsense knowledge, and extend the controllable generation method called NADO to train an auxiliary head.
  arXiv  Detail & Related papers  (2023-10-25T23:32:12Z)
• BLASER: A Text-Free Speech-to-Speech Translation Evaluation Metric [66.73705349465207]
  End-to-end speech-to-speech translation (S2ST) is generally evaluated with text-based metrics. We propose a text-free evaluation metric for end-to-end S2ST, named BLASER, to avoid the dependency on ASR systems.
  arXiv  Detail & Related papers  (2022-12-16T14:00:26Z)
• Proficiency assessment of L2 spoken English using wav2vec 2.0 [3.4012007729454816]
  We use wav2vec 2.0 for assessing overall and individual aspects of proficiency on two small datasets. We find that this approach significantly outperforms the BERT-based baseline system trained on ASR and manual transcriptions used for comparison.
  arXiv  Detail & Related papers  (2022-10-24T12:36:49Z)
• COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation [56.520470678876656]
  bias inherent in user written text can associate different levels of linguistic quality with users' protected attributes. We introduce a general framework to achieve measure-specific counterfactual fairness in explanation generation.
  arXiv  Detail & Related papers  (2022-10-14T02:29:10Z)
• Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition [14.82259273703819]
  We present results using fine-tuned GPT, GPT-2 and their combination for automatic speech recognition (ASR) A conversion method is proposed to compute the correct language prior probability based on bidirectional LM outputs. The proposed conversion for language prior probabilities enables BERT to receive an extra 3% relative WERR.
  arXiv  Detail & Related papers  (2021-07-29T16:53:37Z)
• Abstractive Summarization of Spoken and Written Instructions with BERT [66.14755043607776]
  We present the first application of the BERTSum model to conversational language. We generate abstractive summaries of narrated instructional videos across a wide variety of topics. We envision this integrated as a feature in intelligent virtual assistants, enabling them to summarize both written and spoken instructional content upon request.
  arXiv  Detail & Related papers  (2020-08-21T20:59:34Z)
• Unsupervised Paraphrase Generation using Pre-trained Language Models [0.0]
  OpenAI's GPT-2 is notable for its capability to generate fluent, well formulated, grammatically consistent text. We leverage this generation capability of GPT-2 to generate paraphrases without any supervision from labelled data. Our experiments show that paraphrases generated with our model are of good quality, are diverse and improves the downstream task performance when used for data augmentation.
  arXiv  Detail & Related papers  (2020-06-09T19:40:19Z)
• Assessing Discourse Relations in Language Generation from GPT-2 [37.30382375828105]
  GPT-2 is suited for generation tasks given its left-to-right language modeling objective. We study the validity of explicit discourse relations in GPT-2's outputs under both organic generation and fine-tuned scenarios.
  arXiv  Detail & Related papers  (2020-04-26T23:29:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.