Multimodal Coordinated Online Behavior: Trade-offs and Strategies

• URL: http://arxiv.org/abs/2507.12108v2
• Date: Tue, 22 Jul 2025 08:38:15 GMT
• Title: Multimodal Coordinated Online Behavior: Trade-offs and Strategies
• Authors: Lorenzo Mannocci, Stefano Cresci, Matteo Magnani, Anna Monreale, Maurizio Tesconi,
• Abstract summary: Coordinated online behavior has become a key focus in digital ecosystem analysis.<n>Traditional methods often rely on monomodal approaches, focusing on single types of interactions.<n>This study compares different ways of operationalizing the detection of multimodal coordinated behavior.
• Score: 1.9651052909588413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Coordinated online behavior, which spans from beneficial collective actions to harmful manipulation such as disinformation campaigns, has become a key focus in digital ecosystem analysis. Traditional methods often rely on monomodal approaches, focusing on single types of interactions like co-retweets or co-hashtags, or consider multiple modalities independently of each other. However, these approaches may overlook the complex dynamics inherent in multimodal coordination. This study compares different ways of operationalizing the detection of multimodal coordinated behavior. It examines the trade-off between weakly and strongly integrated multimodal models, highlighting the balance between capturing broader coordination patterns and identifying tightly coordinated behavior. By comparing monomodal and multimodal approaches, we assess the unique contributions of different data modalities and explore how varying implementations of multimodality impact detection outcomes. Our findings reveal that not all the modalities provide distinct insights, but that with a multimodal approach we can get a more comprehensive understanding of coordination dynamics. This work enhances the ability to detect and analyze coordinated online behavior, offering new perspectives for safeguarding the integrity of digital platforms.

Related papers

• Detecting Coordinated Activities Through Temporal, Multiplex, and Collaborative Analysis [2.7415651415305597]
  coordinated campaigns are better characterized by evidence of similar temporal behavioral patterns.<n>We propose a framework to model complex coordination patterns across multiple online modalities.<n>Our results demonstrate that a multiplex time-aware model excels in the identification of coordinating groups.
  arXiv  Detail & Related papers  (2025-12-22T18:53:43Z)
• Offline Multi-agent Reinforcement Learning via Score Decomposition [51.23590397383217]
  offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts.<n>This work is the first work to explicitly address the distributional gap between offline and online MARL.
  arXiv  Detail & Related papers  (2025-05-09T11:42:31Z)
• Asymmetric Reinforcing against Multi-modal Representation Bias [59.685072206359855]
  We propose an Asymmetric Reinforcing method against Multimodal representation bias (ARM)<n>Our ARM dynamically reinforces the weak modalities while maintaining the ability to represent dominant modalities through conditional mutual information.<n>We have significantly improved the performance of multimodal learning, making notable progress in mitigating imbalanced multimodal learning.
  arXiv  Detail & Related papers  (2025-01-02T13:00:06Z)
• On the Comparison between Multi-modal and Single-modal Contrastive Learning [50.74988548106031]
  We introduce a theoretical foundation for understanding the differences between multi-modal and single-modal contrastive learning. We identify the critical factor, which is the signal-to-noise ratio (SNR), that impacts the generalizability in downstream tasks of both multi-modal and single-modal contrastive learning. Our analysis provides a unified framework that can characterize the optimization and generalization of both single-modal and multi-modal contrastive learning.
  arXiv  Detail & Related papers  (2024-11-05T06:21:17Z)
• What to align in multimodal contrastive learning? [7.7439394183358745]
  We introduce Contrastive MultiModal learning strategy that enables the communication between modalities in a single multimodal space.<n>Our theoretical analysis shows that shared, synergistic and unique terms of information naturally emerge from this formulation.<n>In the latter, CoMM learns complex multimodal interactions and achieves state-of-the-art results on the seven multimodal benchmarks.
  arXiv  Detail & Related papers  (2024-09-11T16:42:22Z)
• Beyond Unimodal Learning: The Importance of Integrating Multiple Modalities for Lifelong Learning [23.035725779568587]
  We study the role and interactions of multiple modalities in mitigating forgetting in deep neural networks (DNNs) Our findings demonstrate that leveraging multiple views and complementary information from multiple modalities enables the model to learn more accurate and robust representations. We propose a method for integrating and aligning the information from different modalities by utilizing the relational structural similarities between the data points in each modality.
  arXiv  Detail & Related papers  (2024-05-04T22:02:58Z)
• Multimodal Representation Learning by Alternating Unimodal Adaptation [73.15829571740866]
  We propose MLA (Multimodal Learning with Alternating Unimodal Adaptation) to overcome challenges where some modalities appear more dominant than others during multimodal learning. MLA reframes the conventional joint multimodal learning process by transforming it into an alternating unimodal learning process. It captures cross-modal interactions through a shared head, which undergoes continuous optimization across different modalities. Experiments are conducted on five diverse datasets, encompassing scenarios with complete modalities and scenarios with missing modalities.
  arXiv  Detail & Related papers  (2023-11-17T18:57:40Z)
• Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding [62.70450216120704]
  Unsupervised pre-training has shown great success in skeleton-based action understanding. We propose a Unified Multimodal Unsupervised Representation Learning framework, called UmURL. UmURL exploits an efficient early-fusion strategy to jointly encode the multi-modal features in a single-stream manner.
  arXiv  Detail & Related papers  (2023-11-06T13:56:57Z)
• Enhancing multimodal cooperation via sample-level modality valuation [10.677997431505815]
  We introduce a sample-level modality valuation metric to evaluate the contribution of each modality for each sample. Via modality valuation we observe that modality discrepancy indeed could be different at sample-level beyond the global contribution discrepancy at dataset-level. Our methods reasonably observe the fine-grained uni-modal contribution and achieve considerable improvement.
  arXiv  Detail & Related papers  (2023-09-12T14:16:34Z)
• Learning Unseen Modality Interaction [54.23533023883659]
  Multimodal learning assumes all modality combinations of interest are available during training to learn cross-modal correspondences. We pose the problem of unseen modality interaction and introduce a first solution. It exploits a module that projects the multidimensional features of different modalities into a common space with rich information preserved.
  arXiv  Detail & Related papers  (2023-06-22T10:53:10Z)
• Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications [90.6849884683226]
  We study the challenge of interaction quantification in a semi-supervised setting with only labeled unimodal data. Using a precise information-theoretic definition of interactions, our key contribution is the derivation of lower and upper bounds. We show how these theoretical results can be used to estimate multimodal model performance, guide data collection, and select appropriate multimodal models for various tasks.
  arXiv  Detail & Related papers  (2023-06-07T15:44:53Z)
• Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos [58.93586436289648]
  We propose a multi-scale cooperative multimodal transformer (MCMulT) architecture for multimodal sentiment analysis. Our model outperforms existing approaches on unaligned multimodal sequences and has strong performance on aligned multimodal sequences.
  arXiv  Detail & Related papers  (2022-06-16T07:47:57Z)
• SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities [7.9602600629569285]
  We use bf SHapley vbf Alue-based bf PErceptual (SHAPE) scores to measure the marginal contribution of individual modalities and the degree of cooperation across modalities. Our experiments suggest that for some tasks where different modalities are complementary, the multi-modal models still tend to use the dominant modality alone. We hope our scores can help improve the understanding of how the present multi-modal models operate on different modalities and encourage more sophisticated methods of integrating multiple modalities.
  arXiv  Detail & Related papers  (2022-04-30T16:35:40Z)
• Self-Supervised Multimodal Domino: in Search of Biomarkers for Alzheimer's Disease [19.86082635340699]
  We propose a taxonomy of all reasonable ways to organize self-supervised representation-learning algorithms. We first evaluate models on toy multimodal MNIST datasets and then apply them to a multimodal neuroimaging dataset with Alzheimer's disease patients. Results show that the proposed approach outperforms previous self-supervised encoder-decoder methods.
  arXiv  Detail & Related papers  (2020-12-25T20:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.