Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors

• URL: http://arxiv.org/abs/2507.05939v1
• Date: Tue, 08 Jul 2025 12:38:19 GMT
• Title: Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors
• Abstract summary: Multimodal Misinformation Detection (MMD) becomes an active topic in the community to automatically identify misinformation.<n>Previous MMD methods focus on supervising detectors by collecting offline data.<n>New events always continually emerge, making MMD models trained on offline data consistently outdated and ineffective.
• Score: 32.97594412075928
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, misinformation articles, especially multimodal ones, are widely spread on social media platforms and cause serious negative effects. To control their propagation, Multimodal Misinformation Detection (MMD) becomes an active topic in the community to automatically identify misinformation. Previous MMD methods focus on supervising detectors by collecting offline data. However, in real-world scenarios, new events always continually emerge, making MMD models trained on offline data consistently outdated and ineffective. To address this issue, training MMD models under online data streams is an alternative, inducing an emerging task named continual MMD. Unfortunately, it is hindered by two major challenges. First, training on new data consistently decreases the detection performance on past data, named past knowledge forgetting. Second, the social environment constantly evolves over time, affecting the generalization on future data. To alleviate these challenges, we propose to remember past knowledge by isolating interference between event-specific parameters with a Dirichlet process-based mixture-of-expert structure, and anticipate future environmental distributions by learning a continuous-time dynamics model. Accordingly, we induce a new continual MMD method DAEDCMD. Extensive experiments demonstrate that DAEDCMD can consistently and significantly outperform the compared methods, including six MMD baselines and three continual learning methods.

Related papers

• Harmful Visual Content Manipulation Matters in Misinformation Detection Under Multimedia Scenarios [24.832076026977873]
  Multimodal Misinformation Detection is a research task known as Multimodal Misinformation Detection (MMD)<n>Recent research has shown that manipulated features within visual content in social media articles serve as valuable clues for MMD.<n>In this study, we aim to identify such multimodal misinformation by capturing two types of features: manipulation features, which represent if visual content has been manipulated, and intention features, which assess the nature of these manipulations.
  arXiv  Detail & Related papers  (2026-03-22T04:37:04Z)
• Multimodal Event Detection: Current Approaches and Defining the New Playground through LLMs and VLMs [2.2178611170510694]
  We study the challenges of detecting events on social media, where traditional unimodal systems struggle due to the rapid and multimodal nature of data dissemination.<n>We employ a range of models, including unimodal ModernBERT and ConvNeXt-V2, multimodal fusion techniques, and advanced generative models like GPT-4o, and LLaVA.
  arXiv  Detail & Related papers  (2025-05-16T04:07:21Z)
• Continual Multimodal Contrastive Learning [70.60542106731813]
  Multimodal contrastive learning (MCL) advances in aligning different modalities and generating multimodal representations in a joint space.<n>However, a critical yet often overlooked challenge remains: multimodal data is rarely collected in a single process, and training from scratch is computationally expensive.<n>In this paper, we formulate CMCL through two specialized principles of stability and plasticity.<n>We theoretically derive a novel optimization-based method, which projects updated gradients from dual sides onto subspaces where any gradient is prevented from interfering with the previously learned knowledge.
  arXiv  Detail & Related papers  (2025-03-19T07:57:08Z)
• Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning [79.46570165281084]
  We propose a Multi-Stage Knowledge Integration network (MulKI) to emulate the human learning process in distillation methods. MulKI achieves this through four stages, including Eliciting Ideas, Adding New Ideas, Distinguishing Ideas, and Making Connections. Our method demonstrates significant improvements in maintaining zero-shot capabilities while supporting continual learning across diverse downstream tasks.
  arXiv  Detail & Related papers  (2024-11-11T07:36:19Z)
• Deep Multimodal Learning with Missing Modality: A Survey [12.873458712005037]
  Multimodal learning techniques designed to handle missing modalities can mitigate this. This survey reviews recent progress in Multimodal Learning with Missing Modality (MLMM)
  arXiv  Detail & Related papers  (2024-09-12T08:15:39Z)
• A Practitioner's Guide to Continual Multimodal Pretraining [83.63894495064855]
  Multimodal foundation models serve numerous applications at the intersection of vision and language.<n>To keep models updated, research into continual pretraining mainly explores scenarios with either infrequent, indiscriminate updates on large-scale new data, or frequent, sample-level updates.<n>We introduce FoMo-in-Flux, a continual multimodal pretraining benchmark with realistic compute constraints and practical deployment requirements.
  arXiv  Detail & Related papers  (2024-08-26T17:59:01Z)
• Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL [57.202733701029594]
  We propose Decision Mamba, a novel multi-grained state space model (SSM) with a self-evolving policy learning strategy.<n>To address these challenges, we propose Decision Mamba, a novel multi-grained state space model (SSM) with a self-evolving policy learning strategy.<n>To mitigate the overfitting issue on noisy trajectories, a self-evolving policy is proposed by using progressive regularization.
  arXiv  Detail & Related papers  (2024-06-08T10:12:00Z)
• Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer [21.57847333976567]
  Multimodal Continual Instruction Tuning (MCIT) enables Multimodal Large Language Models (MLLMs) to meet continuously emerging requirements without expensive retraining. MCIT faces two major obstacles: catastrophic forgetting (where old knowledge is forgotten) and negative forward transfer. We propose Prompt Tuning with Positive Forward Transfer (Fwd-Prompt) to address these issues.
  arXiv  Detail & Related papers  (2024-01-17T12:44:17Z)
• Multimodal Representation Learning by Alternating Unimodal Adaptation [73.15829571740866]
  We propose MLA (Multimodal Learning with Alternating Unimodal Adaptation) to overcome challenges where some modalities appear more dominant than others during multimodal learning. MLA reframes the conventional joint multimodal learning process by transforming it into an alternating unimodal learning process. It captures cross-modal interactions through a shared head, which undergoes continuous optimization across different modalities. Experiments are conducted on five diverse datasets, encompassing scenarios with complete modalities and scenarios with missing modalities.
  arXiv  Detail & Related papers  (2023-11-17T18:57:40Z)
• Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning [101.66860222415512]
  Multi-Task Diffusion Model (textscMTDiff) is a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis. For generative planning, we find textscMTDiff outperforms state-of-the-art algorithms across 50 tasks on Meta-World and 8 maps on Maze2D.
  arXiv  Detail & Related papers  (2023-05-29T05:20:38Z)
• MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition [73.80088682784587]
  "Multimodal Generalization" (MMG) aims to study how systems can generalize when data from certain modalities is limited or even completely missing. MMG consists of two novel scenarios, designed to support security, and efficiency considerations in real-world applications. New fusion module with modality dropout training, contrastive-based alignment training, and a novel cross-modal loss for better few-shot performance.
  arXiv  Detail & Related papers  (2023-05-12T03:05:40Z)
• VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias [17.107961913114778]
  multimodal misinformation is a growing problem on social media platforms. In this study, we investigate and identify the presence of unimodal bias in widely-used MMD benchmarks. We introduce a new method -- termed Crossmodal HArd Synthetic MisAlignment (CHASMA) -- for generating realistic synthetic training data.
  arXiv  Detail & Related papers  (2023-04-27T12:28:29Z)
• Online Continual Learning via the Meta-learning Update with Multi-scale Knowledge Distillation and Data Augmentation [4.109784267309124]
  Continual learning aims to rapidly and continually learn the current task from a sequence of tasks. One common limitation of this method is the data imbalance between the previous and current tasks. We propose a novel framework called Meta-learning update via Multi-scale Knowledge Distillation and Data Augmentation.
  arXiv  Detail & Related papers  (2022-09-12T10:03:53Z)
• Lifelong Intent Detection via Multi-Strategy Rebalancing [18.424132535727217]
  In this paper, we propose Lifelong Intent Detection (LID), which continually trains an ID model on new data to learn newly emerging intents. Existing lifelong learning methods usually suffer from a serious imbalance between old and new data in the LID task. We propose a novel lifelong learning method, Multi-Strategy Rebalancing (MSR), which consists of cosine normalization, hierarchical knowledge distillation, and inter-class margin loss.
  arXiv  Detail & Related papers  (2021-08-10T04:35:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.