Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech

• URL: http://arxiv.org/abs/2301.11579v1
• Date: Fri, 27 Jan 2023 07:59:31 GMT
• Title: Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech
• Authors: Jarod Govers, Philip Feldman, Aaron Dant, Panos Patros
• Abstract summary: This study provides the first cross-examination of textual, network visual approaches to detecting extremist content. We identify consensus-driven ERH definitions and propose solutions, particularly due to the lack of research in Oceania/Australasia. We conclude with vital recommendations for ERH mining researchers and propose roadmap with guidelines for researchers, industries, and governments to enable safer cyberspace.
• Score: 1.0323063834827415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social media is a modern person's digital voice to project and engage with new ideas and mobilise communities $\unicode{x2013}$ a power shared with extremists. Given the societal risks of unvetted content-moderating algorithms for Extremism, Radicalisation, and Hate speech (ERH) detection, responsible software engineering must understand the who, what, when, where, and why such models are necessary to protect user safety and free expression. Hence, we propose and examine the unique research field of ERH context mining to unify disjoint studies. Specifically, we evaluate the start-to-finish design process from socio-technical definition-building and dataset collection strategies to technical algorithm design and performance. Our 2015-2021 51-study Systematic Literature Review (SLR) provides the first cross-examination of textual, network, and visual approaches to detecting extremist affiliation, hateful content, and radicalisation towards groups and movements. We identify consensus-driven ERH definitions and propose solutions to existing ideological and geographic biases, particularly due to the lack of research in Oceania/Australasia. Our hybridised investigation on Natural Language Processing, Community Detection, and visual-text models demonstrates the dominating performance of textual transformer-based algorithms. We conclude with vital recommendations for ERH context mining researchers and propose an uptake roadmap with guidelines for researchers, industries, and governments to enable a safer cyberspace.

Related papers

• Modes of Analyzing Disinformation Narratives With AI/ML/Text Mining to Assist in Mitigating the Weaponization of Social Media [0.8287206589886879]
  This paper highlights the developing need for quantitative modes for capturing and monitoring malicious communication in social media. There has been a deliberate "weaponization" of messaging through the use of social networks including by politically oriented entities both state sponsored and privately run. Despite attempts to introduce moderation on major platforms like Facebook and X/Twitter, there are now established alternative social networks that offer completely unmoderated spaces.
  arXiv  Detail & Related papers  (2024-05-25T00:02:14Z)
• A Survey of Generative Search and Recommendation in the Era of Large Language Models [125.26354486027408]
  generative search (retrieval) and recommendation aims to address the matching problem in a generative manner. Superintelligent generative large language models have sparked a new paradigm in search and recommendation.
  arXiv  Detail & Related papers  (2024-04-25T17:58:17Z)
• Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models [52.24001776263608]
  This comprehensive survey delves into the recent strides in HS moderation. We highlight the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) We identify existing gaps in research, particularly in the context of underrepresented languages and cultures.
  arXiv  Detail & Related papers  (2024-01-30T03:51:44Z)
• Combatting Human Trafficking in the Cyberspace: A Natural Language Processing-Based Methodology to Analyze the Language in Online Advertisements [55.2480439325792]
  This project tackles the pressing issue of human trafficking in online C2C marketplaces through advanced Natural Language Processing (NLP) techniques. We introduce a novel methodology for generating pseudo-labeled datasets with minimal supervision, serving as a rich resource for training state-of-the-art NLP models. A key contribution is the implementation of an interpretability framework using Integrated Gradients, providing explainable insights crucial for law enforcement.
  arXiv  Detail & Related papers  (2023-11-22T02:45:01Z)
• Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey [97.33926242130732]
  Large Language Models (LLMs) have revolutionized the domain of natural language processing (NLP) with remarkable capabilities of generating human-like text responses. Despite these advancements, several works in the existing literature have raised serious concerns about the potential misuse of LLMs. To address these concerns, a consensus among the research community is to develop algorithmic solutions to detect AI-generated text.
  arXiv  Detail & Related papers  (2023-10-23T18:11:32Z)
• Large Language Models for Information Retrieval: A Survey [58.30439850203101]
  Information retrieval has evolved from term-based methods to its integration with advanced neural models. Recent research has sought to leverage large language models (LLMs) to improve IR systems. We delve into the confluence of LLMs and IR systems, including crucial aspects such as query rewriters, retrievers, rerankers, and readers.
  arXiv  Detail & Related papers  (2023-08-14T12:47:22Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology [20.81930455526026]
  We propose a fully unsupervised method to detect bias in contextualized embeddings. We show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.
  arXiv  Detail & Related papers  (2022-12-14T23:31:14Z)
• Community as a Vague Operator: Epistemological Questions for a Critical Heuristics of Community Detection Algorithms [0.0]
  We aim to analyse the nature and consequences of what figures in network science as patterns of nodes and edges called 'communities' Disentangling different lineages in network science allows us to contextualise the founding account of 'community' popularised by Michelle Girvan and Mark Newman in 2002. We argue that 'community' can act as a real abstraction with the power to reshape social relations such as producing echo chambers in social networking sites.
  arXiv  Detail & Related papers  (2022-10-06T08:46:57Z)
• Adversarial Attacks and Defenses for Social Network Text Processing Applications: Techniques, Challenges and Future Research Directions [7.84287273674205]
  We provide a review of the main approaches for adversarial attacks and defenses in the context of social media applications. In detail, we cover on six key applications, namely (i) rumors detection, (ii) satires detection, (iii) clickbait & spams identification, (iv) hate speech detection, (v)misinformation detection, and (vi) sentiment analysis.
  arXiv  Detail & Related papers  (2021-10-26T19:33:40Z)
• A systematic review of Hate Speech automatic detection using Natural Language Processing [0.45687771576879593]
  This paper provides a systematic review of literature in this field, with a focus on natural language processing and deep learning technologies. Existing surveys, limitations, and future research directions are extensively discussed.
  arXiv  Detail & Related papers  (2021-05-22T21:48:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.