Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech

• URL: http://arxiv.org/abs/2301.11579v1
• Date: Fri, 27 Jan 2023 07:59:31 GMT
• Title: Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech
• Authors: Jarod Govers, Philip Feldman, Aaron Dant, Panos Patros
• Abstract summary: This study provides the first cross-examination of textual, network visual approaches to detecting extremist content. We identify consensus-driven ERH definitions and propose solutions, particularly due to the lack of research in Oceania/Australasia. We conclude with vital recommendations for ERH mining researchers and propose roadmap with guidelines for researchers, industries, and governments to enable safer cyberspace.
• Score: 1.0323063834827415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social media is a modern person's digital voice to project and engage with new ideas and mobilise communities $\unicode{x2013}$ a power shared with extremists. Given the societal risks of unvetted content-moderating algorithms for Extremism, Radicalisation, and Hate speech (ERH) detection, responsible software engineering must understand the who, what, when, where, and why such models are necessary to protect user safety and free expression. Hence, we propose and examine the unique research field of ERH context mining to unify disjoint studies. Specifically, we evaluate the start-to-finish design process from socio-technical definition-building and dataset collection strategies to technical algorithm design and performance. Our 2015-2021 51-study Systematic Literature Review (SLR) provides the first cross-examination of textual, network, and visual approaches to detecting extremist affiliation, hateful content, and radicalisation towards groups and movements. We identify consensus-driven ERH definitions and propose solutions to existing ideological and geographic biases, particularly due to the lack of research in Oceania/Australasia. Our hybridised investigation on Natural Language Processing, Community Detection, and visual-text models demonstrates the dominating performance of textual transformer-based algorithms. We conclude with vital recommendations for ERH context mining researchers and propose an uptake roadmap with guidelines for researchers, industries, and governments to enable a safer cyberspace.

Related papers

• AI Deception: Risks, Dynamics, and Controls [153.71048309527225]
  This project provides a comprehensive and up-to-date overview of the AI deception field.<n>We identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception.<n>We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment.
  arXiv  Detail & Related papers  (2025-11-27T16:56:04Z)
• Deception Decoder: Proposing a Human-Focused Framework for Identifying AI-Generated Content on Social Media [0.0]
  Generative AI (GenAI) poses a substantial threat to the integrity of information within the contemporary public sphere.<n>This dissertation proposes the Deception Decoder; a framework designed to support general users in identifying AI-generated misinformation and disinformation across text, image, and video.
  arXiv  Detail & Related papers  (2025-11-03T15:55:27Z)
• The Open Syndrome Definition [61.0983330391914]
  We propose the first open, machine-readable format for representing case and syndrome definitions.<n>We introduce the first comprehensive dataset of standardized case definitions and tools to convert existing human-readable definitions into machine-readable formats.<n>The Open Syndrome Definition format enables consistent, scalable use of case definitions across systems, unlocking AI's potential to strengthen public health preparedness and response.
  arXiv  Detail & Related papers  (2025-09-29T19:41:54Z)
• Algorithms for Adversarially Robust Deep Learning [58.656107500646364]
  We discuss recent progress toward designing algorithms that exhibit desirable robustness properties.<n>We present new algorithms that achieve state-of-the-art generalization in medical imaging, molecular identification, and image classification.<n>We propose new attacks and defenses, which represent the frontier of progress toward designing robust language-based agents.
  arXiv  Detail & Related papers  (2025-09-23T14:48:58Z)
• A Practical Synthesis of Detecting AI-Generated Textual, Visual, and Audio Content [2.3543188414616534]
  Advances in AI-generated content have led to wide adoption of large language models, diffusion-based visual generators, and synthetic audio tools. These developments raise concerns about misinformation, copyright infringement, security threats, and the erosion of public trust. This paper explores an extensive range of methods designed to detect and mitigate AI-generated textual, visual, and audio content.
  arXiv  Detail & Related papers  (2025-04-02T23:27:55Z)
• A Survey of Stance Detection on Social Media: New Directions and Perspectives [50.27382951812502]
  stance detection has emerged as a crucial subfield within affective computing. Recent years have seen a surge of research interest in developing effective stance detection methods. This paper provides a comprehensive survey of stance detection techniques on social media.
  arXiv  Detail & Related papers  (2024-09-24T03:06:25Z)
• Modes of Analyzing Disinformation Narratives With AI/ML/Text Mining to Assist in Mitigating the Weaponization of Social Media [0.8287206589886879]
  This paper highlights the developing need for quantitative modes for capturing and monitoring malicious communication in social media. There has been a deliberate "weaponization" of messaging through the use of social networks including by politically oriented entities both state sponsored and privately run. Despite attempts to introduce moderation on major platforms like Facebook and X/Twitter, there are now established alternative social networks that offer completely unmoderated spaces.
  arXiv  Detail & Related papers  (2024-05-25T00:02:14Z)
• A Survey of Generative Search and Recommendation in the Era of Large Language Models [125.26354486027408]
  generative search (retrieval) and recommendation aims to address the matching problem in a generative manner. Superintelligent generative large language models have sparked a new paradigm in search and recommendation.
  arXiv  Detail & Related papers  (2024-04-25T17:58:17Z)
• Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models [52.24001776263608]
  This comprehensive survey delves into the recent strides in HS moderation. We highlight the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) We identify existing gaps in research, particularly in the context of underrepresented languages and cultures.
  arXiv  Detail & Related papers  (2024-01-30T03:51:44Z)
• Combatting Human Trafficking in the Cyberspace: A Natural Language Processing-Based Methodology to Analyze the Language in Online Advertisements [55.2480439325792]
  This project tackles the pressing issue of human trafficking in online C2C marketplaces through advanced Natural Language Processing (NLP) techniques. We introduce a novel methodology for generating pseudo-labeled datasets with minimal supervision, serving as a rich resource for training state-of-the-art NLP models. A key contribution is the implementation of an interpretability framework using Integrated Gradients, providing explainable insights crucial for law enforcement.
  arXiv  Detail & Related papers  (2023-11-22T02:45:01Z)
• Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey [97.33926242130732]
  Large Language Models (LLMs) have revolutionized the domain of natural language processing (NLP) with remarkable capabilities of generating human-like text responses. Despite these advancements, several works in the existing literature have raised serious concerns about the potential misuse of LLMs. To address these concerns, a consensus among the research community is to develop algorithmic solutions to detect AI-generated text.
  arXiv  Detail & Related papers  (2023-10-23T18:11:32Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology [20.81930455526026]
  We propose a fully unsupervised method to detect bias in contextualized embeddings. We show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.
  arXiv  Detail & Related papers  (2022-12-14T23:31:14Z)
• Community as a Vague Operator: Epistemological Questions for a Critical Heuristics of Community Detection Algorithms [0.0]
  We aim to analyse the nature and consequences of what figures in network science as patterns of nodes and edges called 'communities' Disentangling different lineages in network science allows us to contextualise the founding account of 'community' popularised by Michelle Girvan and Mark Newman in 2002. We argue that 'community' can act as a real abstraction with the power to reshape social relations such as producing echo chambers in social networking sites.
  arXiv  Detail & Related papers  (2022-10-06T08:46:57Z)
• Adversarial Attacks and Defenses for Social Network Text Processing Applications: Techniques, Challenges and Future Research Directions [7.84287273674205]
  We provide a review of the main approaches for adversarial attacks and defenses in the context of social media applications. In detail, we cover on six key applications, namely (i) rumors detection, (ii) satires detection, (iii) clickbait & spams identification, (iv) hate speech detection, (v)misinformation detection, and (vi) sentiment analysis.
  arXiv  Detail & Related papers  (2021-10-26T19:33:40Z)
• A systematic review of Hate Speech automatic detection using Natural Language Processing [0.45687771576879593]
  This paper provides a systematic review of literature in this field, with a focus on natural language processing and deep learning technologies. Existing surveys, limitations, and future research directions are extensively discussed.
  arXiv  Detail & Related papers  (2021-05-22T21:48:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.