IOHunter: Graph Foundation Model to Uncover Online Information Operations

• URL: http://arxiv.org/abs/2412.14663v2
• Date: Mon, 03 Mar 2025 15:32:17 GMT
• Title: IOHunter: Graph Foundation Model to Uncover Online Information Operations
• Authors: Marco Minici, Luca Luceri, Francesco Fabbri, Emilio Ferrara,
• Abstract summary: We introduce a methodology designed to identify users orchestrating information operations, a.k.a. IO drivers, across various influence campaigns.<n>Our framework, named IOHunter, leverages the combined strengths of Language Models and Graph Neural Networks to improve generalization in supervised, scarcely-supervised, and cross-IO contexts.<n>This research marks a step toward developing Graph Foundation Models specifically tailored for the task of IO detection on social media platforms.
• Score: 8.532129691916348
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Social media platforms have become vital spaces for public discourse, serving as modern agor\`as where a wide range of voices influence societal narratives. However, their open nature also makes them vulnerable to exploitation by malicious actors, including state-sponsored entities, who can conduct information operations (IOs) to manipulate public opinion. The spread of misinformation, false news, and misleading claims threatens democratic processes and societal cohesion, making it crucial to develop methods for the timely detection of inauthentic activity to protect the integrity of online discourse. In this work, we introduce a methodology designed to identify users orchestrating information operations, a.k.a. IO drivers, across various influence campaigns. Our framework, named IOHunter, leverages the combined strengths of Language Models and Graph Neural Networks to improve generalization in supervised, scarcely-supervised, and cross-IO contexts. Our approach achieves state-of-the-art performance across multiple sets of IOs originating from six countries, significantly surpassing existing approaches. This research marks a step toward developing Graph Foundation Models specifically tailored for the task of IO detection on social media platforms.

Related papers

• Extracting Participation in Collective Action from Social Media [0.0]
  We present a novel suite of text classifiers designed to identify expressions of participation in collective action from social media posts. Our classification captures participation and categorizes it into four levels: recognizing collective issues, engaging in calls-to-action, expressing intention of action, and reporting active involvement. Our findings show that smaller language models can reliably detect expressions of participation, and rival larger models in capturing nuanced levels of participation.
  arXiv  Detail & Related papers  (2025-01-13T14:36:41Z)
• Persuasion with Large Language Models: a Survey [49.86930318312291]
  Large Language Models (LLMs) have created new disruptive possibilities for persuasive communication. In areas such as politics, marketing, public health, e-commerce, and charitable giving, such LLM Systems have already achieved human-level or even super-human persuasiveness. Our survey suggests that the current and future potential of LLM-based persuasion poses profound ethical and societal risks.
  arXiv  Detail & Related papers  (2024-11-11T10:05:52Z)
• MisinfoEval: Generative AI in the Era of "Alternative Facts" [50.069577397751175]
  We introduce a framework for generating and evaluating large language model (LLM) based misinformation interventions. We present (1) an experiment with a simulated social media environment to measure effectiveness of misinformation interventions, and (2) a second experiment with personalized explanations tailored to the demographics and beliefs of users. Our findings confirm that LLM-based interventions are highly effective at correcting user behavior.
  arXiv  Detail & Related papers  (2024-10-13T18:16:50Z)
• Modes of Analyzing Disinformation Narratives With AI/ML/Text Mining to Assist in Mitigating the Weaponization of Social Media [0.8287206589886879]
  This paper highlights the developing need for quantitative modes for capturing and monitoring malicious communication in social media. There has been a deliberate "weaponization" of messaging through the use of social networks including by politically oriented entities both state sponsored and privately run. Despite attempts to introduce moderation on major platforms like Facebook and X/Twitter, there are now established alternative social networks that offer completely unmoderated spaces.
  arXiv  Detail & Related papers  (2024-05-25T00:02:14Z)
• Time Series Analysis of Key Societal Events as Reflected in Complex Social Media Data Streams [0.9790236766474201]
  This study investigates narrative evolution on a niche social media platform GAB and an established messaging service Telegram. Our approach is a novel mode to study multiple social media domains to distil key information which may be obscured otherwise. The main findings are: (1) the time line can be deconstructed to provide useful data features allowing for improved interpretation; (2) a methodology is applied which provides a basis for generalization.
  arXiv  Detail & Related papers  (2024-03-11T18:33:56Z)
• Rational Silence and False Polarization: How Viewpoint Organizations and Recommender Systems Distort the Expression of Public Opinion [4.419843514606336]
  We show how platforms impact what observers of online discourse come to believe about community views. We show that signals from ideological organizations encourage an increase in rhetorical intensity, leading to the 'rational silence' of moderate users. We identify practical strategies platforms can implement, such as reducing exposure to signals from ideological organizations.
  arXiv  Detail & Related papers  (2024-03-10T17:02:19Z)
• SoMeLVLM: A Large Vision Language Model for Social Media Processing [78.47310657638567]
  We introduce a Large Vision Language Model for Social Media Processing (SoMeLVLM) SoMeLVLM is a cognitive framework equipped with five key capabilities including knowledge & comprehension, application, analysis, evaluation, and creation. Our experiments demonstrate that SoMeLVLM achieves state-of-the-art performance in multiple social media tasks.
  arXiv  Detail & Related papers  (2024-02-20T14:02:45Z)
• MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection [2.433983268807517]
  Hate speech poses significant social, psychological, and occasionally physical threats to targeted individuals and communities. Current computational linguistic approaches for tackling this phenomenon rely on labelled social media datasets for training. We scrutinized over 60 datasets, selectively integrating those pertinent into MetaHate. Our findings contribute to a deeper understanding of the existing datasets, paving the way for training more robust and adaptable models.
  arXiv  Detail & Related papers  (2024-01-12T11:54:53Z)
• Leveraging Large Language Models to Detect Influence Campaigns in Social Media [9.58546889761175]
  Social media influence campaigns pose significant challenges to public discourse and democracy. Traditional detection methods fall short due to the complexity and dynamic nature of social media. We propose a novel detection method using Large Language Models (LLMs) that incorporates both user metadata and network structures.
  arXiv  Detail & Related papers  (2023-11-14T00:25:09Z)
• Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting [74.68371461260946]
  SocialSense is a framework that induces a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics. Our method surpasses existing state-of-the-art in experimental evaluations for both zero-shot and supervised settings.
  arXiv  Detail & Related papers  (2023-10-20T06:17:02Z)
• ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media [74.93847489218008]
  We present a novel task, identifying manipulation of news on social media, which aims to detect manipulation in social media posts and identify manipulated or inserted information.<n>To study this task, we have proposed a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles.<n>Our analysis demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance.
  arXiv  Detail & Related papers  (2023-05-23T16:40:07Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Social-Media Activity Forecasting with Exogenous Information Signals [0.0]
  Social media platforms present an ideal environment for studying and understanding social behavior. We propose a modeling technique that forecasts topic-specific daily volume of social media activities.
  arXiv  Detail & Related papers  (2021-09-22T20:24:20Z)
• Empowering NGOs in Countering Online Hate Messages [14.767716319266997]
  We introduce a novel ICT platform that NGO operators can use to monitor and analyze social media data, along with a counter-narrative suggestion tool. We test the platform with more than one hundred NGO operators in three countries through qualitative and quantitative evaluation. Results show that NGOs favor the platform solution with the suggestion tool, and that the time required to produce counter-narratives significantly decreases.
  arXiv  Detail & Related papers  (2021-07-06T08:36:24Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.