BLP-2023 Task 2: Sentiment Analysis

• URL: http://arxiv.org/abs/2310.16183v2
• Date: Thu, 22 Feb 2024 02:32:06 GMT
• Title: BLP-2023 Task 2: Sentiment Analysis
• Authors: Md. Arid Hasan, Firoj Alam, Anika Anjum, Shudipta Das, Afiyat Anjum
• Abstract summary: We present an overview of the BLP Sentiment Shared Task, organized as part of the inaugural BLP 2023 workshop. The task is defined as the detection of sentiment in a given piece of social media text. This paper provides a detailed account of the task setup, including dataset development and evaluation setup.
• Score: 7.725694295666573
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present an overview of the BLP Sentiment Shared Task, organized as part of the inaugural BLP 2023 workshop, co-located with EMNLP 2023. The task is defined as the detection of sentiment in a given piece of social media text. This task attracted interest from 71 participants, among whom 29 and 30 teams submitted systems during the development and evaluation phases, respectively. In total, participants submitted 597 runs. However, a total of 15 teams submitted system description papers. The range of approaches in the submitted systems spans from classical machine learning models, fine-tuning pre-trained models, to leveraging Large Language Model (LLMs) in zero- and few-shot settings. In this paper, we provide a detailed account of the task setup, including dataset development and evaluation setup. Additionally, we provide a brief overview of the systems submitted by the participants. All datasets and evaluation scripts from the shared task have been made publicly available for the research community, to foster further research in this domain.

Related papers

• ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content [9.287041393988485]
  We present an overview of the second edition of the ArAIEval shared task, organized as part of the Arabic 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of 14 teams participated in the final evaluation phase, with 6 and 9 teams participating in Tasks 1 and 2, respectively.
  arXiv  Detail & Related papers  (2024-07-05T04:28:46Z)
• SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes [48.83290963506378]
  This paper presents the results of the SHROOM, a shared task focused on detecting hallucinations. We observe a number of key trends in how this approach was tackled. While a majority of the teams did outperform our proposed baseline system, the performances of top-scoring systems are still consistent with a random handling of the more challenging items.
  arXiv  Detail & Related papers  (2024-03-12T15:06:22Z)
• Overview of the PromptCBLUE Shared Task in CHIP2023 [26.56584015791646]
  This paper presents an overview of the PromptC BLUE shared task held in the CHIP-2023 Conference. It provides a good testbed for Chinese open-domain or medical-domain large language models (LLMs) in general medical natural language processing. This paper describes the tasks, the datasets, evaluation metrics, and the top systems for both tasks.
  arXiv  Detail & Related papers  (2023-12-29T09:05:00Z)
• nlpBDpatriots at BLP-2023 Task 2: A Transfer Learning Approach to Bangla Sentiment Analysis [7.3481279783709805]
  In this paper, we discuss the nlpBDpatriots entry to the shared task on Sentiment Analysis of Bangla Social Media Posts. The main objective of this task is to identify the polarity of social media content using a Bangla dataset annotated with positive, neutral, and negative labels. Our best system ranked 12th among 30 teams that participated in the competition.
  arXiv  Detail & Related papers  (2023-11-25T13:58:58Z)
• ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text [41.3267575540348]
  We present an overview of the ArAIEval shared task, organized as part of the first Arabic 2023 conference co-located with EMNLP 2023. ArAIEval offers two tasks over Arabic text: (i) persuasion technique detection, focusing on identifying persuasion techniques in tweets and news articles, and (ii) disinformation detection in binary and multiclass setups over tweets. A total of 20 teams participated in the final evaluation phase, with 14 and 16 teams participating in Tasks 1 and 2, respectively.
  arXiv  Detail & Related papers  (2023-11-06T15:21:19Z)
• Little Giants: Exploring the Potential of Small LLMs as Evaluation Metrics in Summarization in the Eval4NLP 2023 Shared Task [53.163534619649866]
  This paper focuses on assessing the effectiveness of prompt-based techniques to empower Large Language Models to handle the task of quality estimation. We conducted systematic experiments with various prompting techniques, including standard prompting, prompts informed by annotator instructions, and innovative chain-of-thought prompting. Our work reveals that combining these approaches using a "small", open source model (orca_mini_v3_7B) yields competitive results.
  arXiv  Detail & Related papers  (2023-11-01T17:44:35Z)
• Overview of the BioLaySumm 2023 Shared Task on Lay Summarization of Biomedical Research Articles [47.04555835353173]
  This paper presents the results of the shared task on Lay Summarisation of Biomedical Research Articles (BioLaySumm) hosted at the BioNLP Workshop at ACL 2023. The goal of this shared task is to develop abstractive summarisation models capable of generating "lay summaries" In addition to overall results, we report on the setup and insights from the BioLaySumm shared task, which attracted a total of 20 participating teams across both subtasks.
  arXiv  Detail & Related papers  (2023-09-29T15:43:42Z)
• SemEval-2022 Task 7: Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts [1.3586926359715774]
  We describe SemEval-2022 Task 7, a shared task on rating the plausibility of clarifications in instructional texts. The dataset for this task consists of manually clarified how-to guides for which we generated alternative clarifications and collected human plausibility judgements. The task of participating systems was to automatically determine the plausibility of a clarification in the respective context.
  arXiv  Detail & Related papers  (2023-09-21T14:19:04Z)
• ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images [198.35937007558078]
  The competition opened on 30th December, 2022 and closed on 24th March, 2023. There are 35 participants and 91 valid submissions received for Track 1, and 15 participants and 26 valid submissions received for Track 2. According to the performance of the submissions, we believe there is still a large gap on the expected information extraction performance for complex and zero-shot scenarios.
  arXiv  Detail & Related papers  (2023-06-05T22:20:52Z)
• Overview of the TREC 2020 Fair Ranking Track [64.16623297717642]
  This paper provides an overview of the NIST TREC 2020 Fair Ranking track. The central goal of the Fair Ranking track is to provide fair exposure to different groups of authors.
  arXiv  Detail & Related papers  (2021-08-11T10:22:05Z)
• Overview of the TREC 2019 Fair Ranking Track [65.15263872493799]
  The goal of the TREC Fair Ranking track was to develop a benchmark for evaluating retrieval systems in terms of fairness to different content providers. This paper presents an overview of the track, including the task definition, descriptions of the data and the annotation process.
  arXiv  Detail & Related papers  (2020-03-25T21:34:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.