YouTube Comments Decoded: Leveraging LLMs for Low Resource Language Classification

• URL: http://arxiv.org/abs/2411.05039v1
• Date: Wed, 06 Nov 2024 17:58:01 GMT
• Title: YouTube Comments Decoded: Leveraging LLMs for Low Resource Language Classification
• Authors: Aniket Deroy, Subhankar Maity,
• Abstract summary: We introduce a novel gold standard corpus designed for sarcasm and sentiment detection within code-mixed texts. The primary objective of this task is to identify sarcasm and sentiment polarity within a code-mixed dataset of Tamil-English and Malayalam-English comments and posts collected from social media platforms. We experiment with state-of-the-art large language models like GPT-3.5 Turbo via prompting to classify comments into sarcastic or non-sarcastic categories.
• Score: 0.0
• License:
• Abstract: Sarcasm detection is a significant challenge in sentiment analysis, particularly due to its nature of conveying opinions where the intended meaning deviates from the literal expression. This challenge is heightened in social media contexts where code-mixing, especially in Dravidian languages, is prevalent. Code-mixing involves the blending of multiple languages within a single utterance, often with non-native scripts, complicating the task for systems trained on monolingual data. This shared task introduces a novel gold standard corpus designed for sarcasm and sentiment detection within code-mixed texts, specifically in Tamil-English and Malayalam-English languages. The primary objective of this task is to identify sarcasm and sentiment polarity within a code-mixed dataset of Tamil-English and Malayalam-English comments and posts collected from social media platforms. Each comment or post is annotated at the message level for sentiment polarity, with particular attention to the challenges posed by class imbalance, reflecting real-world scenarios.In this work, we experiment with state-of-the-art large language models like GPT-3.5 Turbo via prompting to classify comments into sarcastic or non-sarcastic categories. We obtained a macro-F1 score of 0.61 for Tamil language. We obtained a macro-F1 score of 0.50 for Malayalam language.

Related papers

• Prompt Engineering Using GPT for Word-Level Code-Mixed Language Identification in Low-Resource Dravidian Languages [0.0]
  In multilingual societies like India, text often exhibits code-mixing, blending local languages with English at different linguistic levels. This paper introduces a prompt based method for a shared task aimed at addressing word-level LI challenges in Dravidian languages. In this work, we leveraged GPT-3.5 Turbo to understand whether the large language models is able to correctly classify words into correct categories.
  arXiv  Detail & Related papers  (2024-11-06T16:20:37Z)
• Understanding and Mitigating Language Confusion in LLMs [76.96033035093204]
  We evaluate 15 typologically diverse languages with existing and newly-created English and multilingual prompts. We find that Llama Instruct and Mistral models exhibit high degrees of language confusion. We find that language confusion can be partially mitigated via few-shot prompting, multilingual SFT and preference tuning.
  arXiv  Detail & Related papers  (2024-06-28T17:03:51Z)
• IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languages [62.60787450345489]
  We explore code-mixing in Indonesian with four embedded languages, i.e., English, Sundanese, Javanese, and Malay. Our analysis shows that the pre-training corpus bias affects the model's ability to better handle Indonesian-English code-mixing.
  arXiv  Detail & Related papers  (2023-11-21T07:50:53Z)
• Marathi-English Code-mixed Text Generation [0.0]
  Code-mixing, the blending of linguistic elements from distinct languages to form meaningful sentences, is common in multilingual settings. This research introduces a Marathi-English code-mixed text generation algorithm, assessed with Code Mixing Index (CMI) and Degree of Code Mixing (DCM) metrics.
  arXiv  Detail & Related papers  (2023-09-28T06:51:26Z)
• Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages [47.78634360870564]
  We explore prompting multilingual models to generate code-mixed data for seven languages in South East Asia (SEA) We find that publicly available multilingual instruction-tuned models such as BLOOMZ are incapable of producing texts with phrases or clauses from different languages. ChatGPT exhibits inconsistent capabilities in generating code-mixed texts, wherein its performance varies depending on the prompt template and language pairing.
  arXiv  Detail & Related papers  (2023-03-23T18:16:30Z)
• MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages [76.93265104421559]
  We benchmark code generation from natural language commands extending beyond English. We annotated a total of 896 NL-code pairs in three languages: Spanish, Japanese, and Russian. While the difficulties vary across these three languages, all systems lag significantly behind their English counterparts.
  arXiv  Detail & Related papers  (2022-03-16T04:21:50Z)
• NUIG-Shubhanker@Dravidian-CodeMix-FIRE2020: Sentiment Analysis of Code-Mixed Dravidian text using XLNet [0.0]
  Social media has penetrated into multilingual societies, however most of them use English to be a preferred language for communication. It looks natural for them to mix their cultural language with English during conversations resulting in abundance of multilingual data, call this code-mixed data, available in todays' world. Downstream NLP tasks using such data is challenging due to the semantic nature of it being spread across multiple languages. This paper uses an auto-regressive XLNet model to perform sentiment analysis on code-mixed Tamil-English and Malayalam-English datasets.
  arXiv  Detail & Related papers  (2020-10-15T14:09:02Z)
• Automatic Extraction of Rules Governing Morphological Agreement [103.78033184221373]
  We develop an automated framework for extracting a first-pass grammatical specification from raw text. We focus on extracting rules describing agreement, a morphosyntactic phenomenon at the core of the grammars of many of the world's languages. We apply our framework to all languages included in the Universal Dependencies project, with promising results.
  arXiv  Detail & Related papers  (2020-10-02T18:31:45Z)
• NLP-CIC at SemEval-2020 Task 9: Analysing sentiment in code-switching language using a simple deep-learning classifier [63.137661897716555]
  Code-switching is a phenomenon in which two or more languages are used in the same message. We use a standard convolutional neural network model to predict the sentiment of tweets in a blend of Spanish and English languages.
  arXiv  Detail & Related papers  (2020-09-07T19:57:09Z)
• A Sentiment Analysis Dataset for Code-Mixed Malayalam-English [0.8454131372606295]
  This paper presents a new gold standard corpus for sentiment analysis of code-mixed text in Malayalam-English annotated by voluntary annotators. We use this new corpus to provide the benchmark for sentiment analysis in Malayalam-English code-mixed texts.
  arXiv  Detail & Related papers  (2020-05-30T07:32:37Z)
• Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text [0.9235531183915556]
  We create a code-switched, sentiment-annotated corpus containing 15,744 comment posts from YouTube. In this paper, we describe the process of creating the corpus and assigning polarities. We present inter-annotator agreement and show the results of sentiment analysis trained on this corpus as a benchmark.
  arXiv  Detail & Related papers  (2020-05-30T07:17:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.