ACTI at EVALITA 2023: Overview of the Conspiracy Theory Identification Task

• URL: http://arxiv.org/abs/2307.06954v3
• Date: Sat, 2 Sep 2023 07:12:42 GMT
• Title: ACTI at EVALITA 2023: Overview of the Conspiracy Theory Identification Task
• Authors: Giuseppe Russo, Niklas Stoehr, Manoel Horta Ribeiro
• Abstract summary: The ACTI challenge, based exclusively on comments published on conspiratorial channels of telegram, is divided into two subtasks. A total of fifteen teams participated in the task for a total of 81 submissions. We illustrate the best performing approaches were based on the utilization of large language models.
• Score: 7.36947519345126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conspiracy Theory Identication task is a new shared task proposed for the first time at the Evalita 2023. The ACTI challenge, based exclusively on comments published on conspiratorial channels of telegram, is divided into two subtasks: (i) Conspiratorial Content Classification: identifying conspiratorial content and (ii) Conspiratorial Category Classification about specific conspiracy theory classification. A total of fifteen teams participated in the task for a total of 81 submissions. We illustrate the best performing approaches were based on the utilization of large language models. We finally draw conclusions about the utilization of these models for counteracting the spreading of misinformation in online platforms.

Related papers

• Classifying Conspiratorial Narratives At Scale: False Alarms and Erroneous Connections [4.594855794205588]
  This work establishes a general scheme for classifying discussions related to conspiracy theories. We leverage human-labeled ground truth to train a BERT-based model for classifying online CTs. We present the first large-scale classification study using posts from the most active conspiracy-related Reddit forums.
  arXiv  Detail & Related papers  (2024-03-29T20:29:12Z)
• SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes [48.83290963506378]
  This paper presents the results of the SHROOM, a shared task focused on detecting hallucinations. We observe a number of key trends in how this approach was tackled. While a majority of the teams did outperform our proposed baseline system, the performances of top-scoring systems are still consistent with a random handling of the more challenging items.
  arXiv  Detail & Related papers  (2024-03-12T15:06:22Z)
• ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text [41.3267575540348]
  We present an overview of the ArAIEval shared task, organized as part of the first Arabic 2023 conference co-located with EMNLP 2023. ArAIEval offers two tasks over Arabic text: (i) persuasion technique detection, focusing on identifying persuasion techniques in tweets and news articles, and (ii) disinformation detection in binary and multiclass setups over tweets. A total of 20 teams participated in the final evaluation phase, with 14 and 16 teams participating in Tasks 1 and 2, respectively.
  arXiv  Detail & Related papers  (2023-11-06T15:21:19Z)
• Overview of the BioLaySumm 2023 Shared Task on Lay Summarization of Biomedical Research Articles [47.04555835353173]
  This paper presents the results of the shared task on Lay Summarisation of Biomedical Research Articles (BioLaySumm) hosted at the BioNLP Workshop at ACL 2023. The goal of this shared task is to develop abstractive summarisation models capable of generating "lay summaries" In addition to overall results, we report on the setup and insights from the BioLaySumm shared task, which attracted a total of 20 participating teams across both subtasks.
  arXiv  Detail & Related papers  (2023-09-29T15:43:42Z)
• UPB @ ACTI: Detecting Conspiracies using fine tuned Sentence Transformers [2.0046675283732887]
  We address conspiracy theory detection as proposed by the ACTI @ EVALA 2023 shared task. Our methodology attained F1 scores of 85.71% in the binary classification and 91.23% for the fine-grained conspiracy topic classification, surpassing other competing systems.
  arXiv  Detail & Related papers  (2023-09-28T09:17:20Z)
• The Anatomy of Conspirators: Unveiling Traits using a Comprehensive Twitter Dataset [0.0]
  We present a novel methodology for constructing a Twitter dataset that encompasses accounts engaged in conspiracy-related activities throughout the year 2022. This comprehensive collection effort yielded a total of 15K accounts and 37M tweets extracted from their timelines. We conduct a comparative analysis of the two groups across three dimensions: topics, profiles, and behavioral characteristics.
  arXiv  Detail & Related papers  (2023-08-29T09:35:23Z)
• ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images [198.35937007558078]
  The competition opened on 30th December, 2022 and closed on 24th March, 2023. There are 35 participants and 91 valid submissions received for Track 1, and 15 participants and 26 valid submissions received for Track 2. According to the performance of the submissions, we believe there is still a large gap on the expected information extraction performance for complex and zero-shot scenarios.
  arXiv  Detail & Related papers  (2023-06-05T22:20:52Z)
• Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection [23.280506220186425]
  We present the results of the NLP4IF-2021 shared tasks. Ten teams submitted systems for task 1, and one team participated in task 2. The best systems used pre-trained Transformers and ensembles.
  arXiv  Detail & Related papers  (2021-09-23T06:38:03Z)
• SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning [47.49596196559958]
  This paper introduces the SemEval-2021 shared task 4: Reading of Abstract Meaning (ReCAM) Given a passage and the corresponding question, a participating system is expected to choose the correct answer from five candidates of abstract concepts. Subtask 1 aims to evaluate how well a system can model concepts that cannot be directly perceived in the physical world. Subtask 2 focuses on models' ability in comprehending nonspecific concepts located high in a hypernym hierarchy. Subtask 3 aims to provide some insights into models' generalizability over the two types of abstractness.
  arXiv  Detail & Related papers  (2021-05-31T11:04:17Z)
• The Truth is Out There: Investigating Conspiracy Theories in Text Generation [66.01545519772527]
  We investigate the propensity for language models to generate conspiracy theory text. Our study focuses on testing these models for the elicitation of conspiracy theories. We introduce a new dataset consisting of conspiracy theory topics, machine-generated conspiracy theories, and human-written conspiracy theories.
  arXiv  Detail & Related papers  (2021-01-02T05:47:39Z)
• Online Action Recognition [69.32402131983699]
  Action Unification (AU) and Online Action Recognition through Unification (OARU) are proposed. AU builds on logic unification and generalizes two input actions using weighted partial MaxSAT. OARU recognizes actions accurately with respect to expert knowledge, and shows real-time performance.
  arXiv  Detail & Related papers  (2020-12-14T12:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.