Lifelong Evolution: Collaborative Learning between Large and Small Language Models for Continuous Emergent Fake News Detection

• URL: http://arxiv.org/abs/2506.04739v1
• Date: Thu, 05 Jun 2025 08:17:55 GMT
• Title: Lifelong Evolution: Collaborative Learning between Large and Small Language Models for Continuous Emergent Fake News Detection
• Authors: Ziyi Zhou, Xiaoming Zhang, Litian Zhang, Yibo Zhang, Zhenyu Guan, Chaozhuo Li, Philip S. Yu,
• Abstract summary: Large language models (LLMs) fall short in accurately detecting fake news owing to outdated knowledge and the absence of suitable demonstrations.<n>We propose a novel Continuous Collaborative Emergent Fake News Detection (C$2$EFND) framework to address these challenges.
• Score: 44.64168455850825
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The widespread dissemination of fake news on social media has significantly impacted society, resulting in serious consequences. Conventional deep learning methodologies employing small language models (SLMs) suffer from extensive supervised training requirements and difficulties adapting to evolving news environments due to data scarcity and distribution shifts. Large language models (LLMs), despite robust zero-shot capabilities, fall short in accurately detecting fake news owing to outdated knowledge and the absence of suitable demonstrations. In this paper, we propose a novel Continuous Collaborative Emergent Fake News Detection (C$^2$EFND) framework to address these challenges. The C$^2$EFND framework strategically leverages both LLMs' generalization power and SLMs' classification expertise via a multi-round collaborative learning framework. We further introduce a lifelong knowledge editing module based on a Mixture-of-Experts architecture to incrementally update LLMs and a replay-based continue learning method to ensure SLMs retain prior knowledge without retraining entirely. Extensive experiments on Pheme and Twitter16 datasets demonstrate that C$^2$EFND significantly outperforms existed methods, effectively improving detection accuracy and adaptability in continuous emergent fake news scenarios.

Related papers

• Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models [79.90523648823522]
  Multi-stage continual learning can lead to catastrophic forgetting.<n>This paper evaluates three mitigation strategies-model merging, discounting the LoRA scaling factor, and experience replay.<n>Results show that experience replay is the most effective, with further gains achieved by combining it with other methods.
  arXiv  Detail & Related papers  (2025-05-23T05:50:14Z)
• Collaborative Evolution: Multi-Round Learning Between Large and Small Language Models for Emergent Fake News Detection [12.65676695802598]
  Large language models (LLMs) have fallen short in effectively identifying fake news due to a lack of pertinent demonstrations and the dynamic nature of knowledge.<n>In this paper, a novel framework Multi-Round Collaboration Detection (MRCD) is proposed to address these limitations.<n>Our framework MRCD achieves SOTA results on two real-world datasets Pheme and Twitter16, with accuracy improvements of 7.4% and 12.8% compared to using only SLMs.
  arXiv  Detail & Related papers  (2025-03-27T03:39:26Z)
• LLM Post-Training: A Deep Dive into Reasoning Large Language Models [131.10969986056]
  Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications.<n>Post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations.
  arXiv  Detail & Related papers  (2025-02-28T18:59:54Z)
• Challenges and Innovations in LLM-Powered Fake News Detection: A Synthesis of Approaches and Future Directions [0.0]
  pervasiveness of the dissemination of fake news through social media platforms poses critical risks to the trust of the general public.<n>Recent works include powering the detection using large language model advances in multimodal frameworks.<n>The review further identifies critical gaps in adaptability to dynamic social media trends, real-time, and cross-platform detection capabilities.
  arXiv  Detail & Related papers  (2025-02-01T06:56:17Z)
• A Self-Learning Multimodal Approach for Fake News Detection [35.98977478616019]
  We introduce a self-learning multimodal model for fake news classification.<n>The model leverages contrastive learning, a robust method for feature extraction that operates without requiring labeled data.<n>Our experimental results on a public dataset demonstrate that the proposed model outperforms several state-of-the-art classification approaches.
  arXiv  Detail & Related papers  (2024-12-08T07:41:44Z)
• Detect, Investigate, Judge and Determine: A Knowledge-guided Framework for Few-shot Fake News Detection [50.079690200471454]
  Few-Shot Fake News Detection (FS-FND) aims to distinguish inaccurate news from real ones in extremely low-resource scenarios.<n>This task has garnered increased attention due to the widespread dissemination and harmful impact of fake news on social media.<n>We propose a Dual-perspective Knowledge-guided Fake News Detection (DKFND) model, designed to enhance LLMs from both inside and outside perspectives.
  arXiv  Detail & Related papers  (2024-07-12T03:15:01Z)
• COOL: Comprehensive Knowledge Enhanced Prompt Learning for Domain Adaptive Few-shot Fake News Detection [16.478355864072814]
  We propose COOL, a comprehensive knedge enhanced prOmpt Learning method for domain adaptive few-shot FND.Owl. Specifically, we propose a comprehensive knowledge extraction module to extract both structured and unstructured knowledge that are positively or negatively correlated with news from external sources.
  arXiv  Detail & Related papers  (2024-06-16T09:41:25Z)
• Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors [38.75533934195315]
  Large Language Models (LLMs) are known for their remarkable reasoning and generative capabilities. We introduce a novel, retrieval-augmented LLMs framework--the first of its kind to automatically and strategically extract key evidence from web sources for claim verification. Our framework ensures the acquisition of sufficient, relevant evidence, thereby enhancing performance.
  arXiv  Detail & Related papers  (2024-03-14T00:35:39Z)
• Continual Learning for Large Language Models: A Survey [95.79977915131145]
  Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. This paper surveys recent works on continual learning for LLMs.
  arXiv  Detail & Related papers  (2024-02-02T12:34:09Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.