ThatiAR: Subjectivity Detection in Arabic News Sentences

• URL: http://arxiv.org/abs/2406.05559v1
• Date: Sat, 8 Jun 2024 19:24:17 GMT
• Title: ThatiAR: Subjectivity Detection in Arabic News Sentences
• Authors: Reem Suwaileh, Maram Hasanain, Fatema Hubail, Wajdi Zaghouani, Firoj Alam,
• Abstract summary: This study presents the first large dataset for subjectivity detection in Arabic. It consists of 3.6K manually annotated sentences, and GPT-4o based explanation. We provide an in-depth analysis of the dataset, annotation process, and extensive benchmark results.
• Score: 10.334164786614696
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Detecting subjectivity in news sentences is crucial for identifying media bias, enhancing credibility, and combating misinformation by flagging opinion-based content. It provides insights into public sentiment, empowers readers to make informed decisions, and encourages critical thinking. While research has developed methods and systems for this purpose, most efforts have focused on English and other high-resourced languages. In this study, we present the first large dataset for subjectivity detection in Arabic, consisting of ~3.6K manually annotated sentences, and GPT-4o based explanation. In addition, we included instructions (both in English and Arabic) to facilitate LLM based fine-tuning. We provide an in-depth analysis of the dataset, annotation process, and extensive benchmark results, including PLMs and LLMs. Our analysis of the annotation process highlights that annotators were strongly influenced by their political, cultural, and religious backgrounds, especially at the beginning of the annotation process. The experimental results suggest that LLMs with in-context learning provide better performance. We aim to release the dataset and resources for the community.

Related papers

• Data-Centric AI in the Age of Large Language Models [51.20451986068925]
  This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs) We make the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs. We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization.
  arXiv  Detail & Related papers  (2024-06-20T16:34:07Z)
• Investigating Annotator Bias in Large Language Models for Hate Speech Detection [5.589665886212444]
  This paper delves into the biases present in Large Language Models (LLMs) when annotating hate speech data. Specifically targeting highly vulnerable groups within these categories, we analyze annotator biases. We introduce our custom hate speech detection dataset, HateBiasNet, to conduct this research.
  arXiv  Detail & Related papers  (2024-06-17T00:18:31Z)
• Bridging the Bosphorus: Advancing Turkish Large Language Models through Strategies for Low-Resource Language Adaptation and Benchmarking [1.3716808114696444]
  Large Language Models (LLMs) are becoming crucial across various fields, emphasizing the urgency for high-quality models in underrepresented languages. This study explores the unique challenges faced by low-resource languages, such as data scarcity, model selection, evaluation, and computational limitations.
  arXiv  Detail & Related papers  (2024-05-07T21:58:45Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectic datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• Content-Localization based System for Analyzing Sentiment and Hate Behaviors in Low-Resource Dialectal Arabic: English to Levantine and Gulf [5.2957928879391]
  This paper proposes to localize content of resources in high-resourced languages into under-resourced Arabic dialects. We utilize content-localization based neural machine translation to develop sentiment and hate classifiers for two low-resourced Arabic dialects: Levantine and Gulf. Our findings shed light on the importance of considering the unique nature of dialects within the same language and ignoring the dialectal aspect would lead to misleading analysis.
  arXiv  Detail & Related papers  (2023-11-27T15:37:33Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• Data-Augmented and Retrieval-Augmented Context Enrichment in Chinese Media Bias Detection [16.343223974292908]
  We build a dataset with Chinese news reports about COVID-19 which is annotated by our newly designed system. In Data-Augmented Context Enrichment (DACE), we enlarge the training data; while in Retrieval-Augmented Context Enrichment (RACE), we improve information retrieval methods to select valuable information. Our results show that both methods outperform our baselines, while the RACE methods are more efficient and have more potential.
  arXiv  Detail & Related papers  (2023-11-02T16:29:49Z)
• Demonstrations Are All You Need: Advancing Offensive Content Paraphrasing using In-Context Learning [10.897468059705238]
  Supervised paraphrasers rely heavily on large quantities of labelled data to help preserve meaning and intent. In this paper we aim to assist practitioners in developing usable paraphrasers by exploring In-Context Learning (ICL) with large language models (LLMs) Our study focuses on key factors such as - number and order of demonstrations, exclusion of prompt instruction, and reduction in measured toxicity.
  arXiv  Detail & Related papers  (2023-10-16T16:18:55Z)
• Cross-Lingual Knowledge Editing in Large Language Models [73.12622532088564]
  Knowledge editing has been shown to adapt large language models to new knowledge without retraining from scratch. It is still unknown the effect of source language editing on a different target language. We first collect a large-scale cross-lingual synthetic dataset by translating ZsRE from English to Chinese.
  arXiv  Detail & Related papers  (2023-09-16T11:07:52Z)
• Context-faithful Prompting for Large Language Models [51.194410884263135]
  Large language models (LLMs) encode parametric knowledge about world facts. Their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks. We assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention.
  arXiv  Detail & Related papers  (2023-03-20T17:54:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.