Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach

• URL: http://arxiv.org/abs/2108.08920v2
• Date: Mon, 23 Aug 2021 02:13:56 GMT
• Title: Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach
• Authors: Chuanbo Hu, Minglei Yin, Bin Liu, Xin Li, Yanfang Ye
• Abstract summary: We conduct the first systematic study on fine-grained detection of illicit drug trafficking events (IDTEs) on Instagram. Specifically, our model takes text and image data as the input and combines multimodal information to predict multiple labels of illicit drugs. We have constructed a large-scale dataset MM-IDTE with manually annotated multiple drug labels to support fine-grained detection of illicit drugs.
• Score: 18.223055392013542
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social media such as Instagram and Twitter have become important platforms for marketing and selling illicit drugs. Detection of online illicit drug trafficking has become critical to combat the online trade of illicit drugs. However, the legal status often varies spatially and temporally; even for the same drug, federal and state legislation can have different regulations about its legality. Meanwhile, more drug trafficking events are disguised as a novel form of advertising commenting leading to information heterogeneity. Accordingly, accurate detection of illicit drug trafficking events (IDTEs) from social media has become even more challenging. In this work, we conduct the first systematic study on fine-grained detection of IDTEs on Instagram. We propose to take a deep multimodal multilabel learning (DMML) approach to detect IDTEs and demonstrate its effectiveness on a newly constructed dataset called multimodal IDTE(MM-IDTE). Specifically, our model takes text and image data as the input and combines multimodal information to predict multiple labels of illicit drugs. Inspired by the success of BERT, we have developed a self-supervised multimodal bidirectional transformer by jointly fine-tuning pretrained text and image encoders. We have constructed a large-scale dataset MM-IDTE with manually annotated multiple drug labels to support fine-grained detection of illicit drugs. Extensive experimental results on the MM-IDTE dataset show that the proposed DMML methodology can accurately detect IDTEs even in the presence of special characters and style changes attempting to evade detection.

Related papers

• Learning to Describe for Predicting Zero-shot Drug-Drug Interactions [54.172575323610175]
  Adverse drug-drug interactions can compromise the effectiveness of concurrent drug administration. Traditional computational methods for DDI prediction may fail to capture interactions for new drugs due to the lack of knowledge. We propose TextDDI with a language model-based DDI predictor and a reinforcement learning(RL)-based information selector.
  arXiv  Detail & Related papers  (2024-03-13T09:42:46Z)
• Multi-modal Stance Detection: New Datasets and Model [56.97470987479277]
  We study multi-modal stance detection for tweets consisting of texts and images. We propose a simple yet effective Targeted Multi-modal Prompt Tuning framework (TMPT) TMPT achieves state-of-the-art performance in multi-modal stance detection.
  arXiv  Detail & Related papers  (2024-02-22T05:24:19Z)
• Detecting and Grounding Multi-Modal Media Manipulation and Beyond [93.08116982163804]
  We highlight a new research problem for multi-modal fake media, namely Detecting and Grounding Multi-Modal Media Manipulation (DGM4) DGM4 aims to not only detect the authenticity of multi-modal media, but also ground the manipulated content. We propose a novel HierArchical Multi-modal Manipulation rEasoning tRansformer (HAMMER) to fully capture the fine-grained interaction between different modalities.
  arXiv  Detail & Related papers  (2023-09-25T15:05:46Z)
• MC-DRE: Multi-Aspect Cross Integration for Drug Event/Entity Extraction [19.4567740328955]
  We propose a new multi-aspect cross-integration framework for drug entity/event detection. Our model outperforms all SOTA on two widely used tasks, flat entity detection and discontinuous event extraction.
  arXiv  Detail & Related papers  (2023-08-12T12:03:41Z)
• Unveiling the Potential of Knowledge-Prompted ChatGPT for Enhancing Drug Trafficking Detection on Social Media [30.791563171321062]
  We propose an analytical framework to compose emphknowledge-informed prompts, which serve as the interface that humans can interact with and use LLMs to perform the detection task. Our experimental findings demonstrate that the proposed framework outperforms other baseline language models in terms of drug trafficking detection accuracy. The implications of our research extend to social networks, emphasizing the importance of incorporating prior knowledge and scenario-based prompts into analytical tools to improve online security and public safety.
  arXiv  Detail & Related papers  (2023-07-07T16:15:59Z)
• Detecting and Grounding Multi-Modal Media Manipulation [32.34908534582532]
  We highlight a new research problem for multi-modal fake media, namely Detecting and Grounding Multi-Modal Media Manipulation (DGM4) DGM4 aims to not only detect the authenticity of multi-modal media, but also ground the manipulated content. We propose a novel HierArchical Multi-modal Manipulation rEasoning tRansformer (HAMMER) to fully capture the fine-grained interaction between different modalities.
  arXiv  Detail & Related papers  (2023-04-05T16:20:40Z)
• SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction [127.43571146741984]
  Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery. wet experiments remain the most reliable method, but they are time-consuming and resource-intensive. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. We present the SSM-DTA framework, which incorporates three simple yet highly effective strategies.
  arXiv  Detail & Related papers  (2022-06-20T14:53:25Z)
• Unsupervised Person Re-Identification with Wireless Positioning under Weak Scene Labeling [131.18390399368997]
  We propose to explore unsupervised person re-identification with both visual data and wireless positioning trajectories under weak scene labeling. Specifically, we propose a novel unsupervised multimodal training framework (UMTF), which models the complementarity of visual data and wireless information. Our UMTF contains a multimodal data association strategy (MMDA) and a multimodal graph neural network (MMGN)
  arXiv  Detail & Related papers  (2021-10-29T08:25:44Z)
• Identifying Illicit Drug Dealers on Instagram with Large-scale Multimodal Data Fusion [18.223055392013542]
  Illicit drug trafficking via social media sites such as Instagram has become a severe problem. How to identify illicit drug dealers from social media data has remained a technical challenge. We propose to tackle the problem of illicit drug dealer identification by constructing a large-scale multimodal dataset.
  arXiv  Detail & Related papers  (2021-08-18T04:29:47Z)
• Multimodal Categorization of Crisis Events in Social Media [81.07061295887172]
  We present a new multimodal fusion method that leverages both images and texts as input. In particular, we introduce a cross-attention module that can filter uninformative and misleading components from weak modalities. We show that our method outperforms the unimodal approaches and strong multimodal baselines by a large margin on three crisis-related tasks.
  arXiv  Detail & Related papers  (2020-04-10T06:31:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.