Knowledge-guided Continual Learning for Behavioral Analytics Systems

• URL: http://arxiv.org/abs/2510.22405v2
• Date: Sat, 01 Nov 2025 19:07:18 GMT
• Title: Knowledge-guided Continual Learning for Behavioral Analytics Systems
• Authors: Yasas Senarath, Hemant Purohit,
• Abstract summary: Models that learn to capture this content can experience a decrease in performance over time due to data drift.<n>Fine-tuning such a model over time with new data can be detrimental due to catastrophic forgetting.<n>Replay-based approaches in continual learning offer a simple yet efficient method to update such models.<n>We propose a novel augmentation-based approach to incorporate external knowledge in the replay-based continual learning framework.
• Score: 0.3437656066916039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: User behavior on online platforms is evolving, reflecting real-world changes in how people post, whether it's helpful messages or hate speech. Models that learn to capture this content can experience a decrease in performance over time due to data drift, which can lead to ineffective behavioral analytics systems. However, fine-tuning such a model over time with new data can be detrimental due to catastrophic forgetting. Replay-based approaches in continual learning offer a simple yet efficient method to update such models, minimizing forgetting by maintaining a buffer of important training instances from past learned tasks. However, the main limitation of this approach is the fixed size of the buffer. External knowledge bases can be utilized to overcome this limitation through data augmentation. We propose a novel augmentation-based approach to incorporate external knowledge in the replay-based continual learning framework. We evaluate several strategies with three datasets from prior studies related to deviant behavior classification to assess the integration of external knowledge in continual learning and demonstrate that augmentation helps outperform baseline replay-based approaches.

Related papers

• Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning [51.07663354001582]
  Deep neural networks suffer from catastrophic forgetting, where performance on previous tasks degrades after training on a new task.<n>We present a novel approach to address this challenge, focusing on the intersection of memory-based methods and regularization approaches.<n>We formulate a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods.
  arXiv  Detail & Related papers  (2025-12-01T15:56:00Z)
• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• ReLearn: Unlearning via Learning for Large Language Models [64.2802606302194]
  We propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning.<n>This framework introduces Knowledge Forgetting Rate (KFR) and Knowledge Retention Rate (KRR) to measure knowledge-level preservation.<n>Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality output.
  arXiv  Detail & Related papers  (2025-02-16T16:31:00Z)
• Temporal-Difference Variational Continual Learning [77.92320830700797]
  We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.<n>Our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [52.40798352740857]
  We introduce the Iterative Contrastive Unlearning (ICU) framework, which consists of three core components.<n>A Knowledge Unlearning Induction module targets specific knowledge for removal using an unlearning loss.<n>A Contrastive Learning Enhancement module preserves the model's expressive capabilities against the pure unlearning goal.<n>An Iterative Unlearning Refinement module dynamically adjusts the unlearning process through ongoing evaluation and updates.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• VERSE: Virtual-Gradient Aware Streaming Lifelong Learning with Anytime Inference [36.61783715563126]
  Streaming lifelong learning is a challenging setting of lifelong learning with the goal of continuous learning without forgetting. We introduce a novel approach to lifelong learning, which is streaming (observes each training example only once) We propose a novel emphvirtual gradients based approach for continual representation learning which adapts to each new example while also generalizing well on past data to prevent catastrophic forgetting.
  arXiv  Detail & Related papers  (2023-09-15T07:54:49Z)
• Advancing continual lifelong learning in neural information retrieval: definition, dataset, framework, and empirical evaluation [3.2340528215722553]
  A systematic task formulation of continual neural information retrieval is presented. A comprehensive continual neural information retrieval framework is proposed. Empirical evaluations illustrate that the proposed framework can successfully prevent catastrophic forgetting in neural information retrieval.
  arXiv  Detail & Related papers  (2023-08-16T14:01:25Z)
• Dissecting Continual Learning a Structural and Data Analysis [0.0]
  Continual Learning is a field dedicated to devise algorithms able to achieve lifelong learning. Deep learning methods can attain impressive results when the data modeled does not undergo a considerable distributional shift in subsequent learning sessions. When we expose such systems to this incremental setting, performance drop very quickly.
  arXiv  Detail & Related papers  (2023-01-03T10:37:11Z)
• PIVOT: Prompting for Video Continual Learning [50.80141083993668]
  We introduce PIVOT, a novel method that leverages extensive knowledge in pre-trained models from the image domain. Our experiments show that PIVOT improves state-of-the-art methods by a significant 27% on the 20-task ActivityNet setup.
  arXiv  Detail & Related papers  (2022-12-09T13:22:27Z)
• Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation [39.97128550414934]
  We present a novel class incremental learning approach based on deep neural networks. It continually learns new tasks with limited memory for storing examples in the previous tasks. Our algorithm is based on knowledge distillation and provides a principled way to maintain the representations of old models.
  arXiv  Detail & Related papers  (2022-04-02T16:30:04Z)
• Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data [101.6195176510611]
  "Online" continual learning enables evaluating both information retention and online learning efficacy. In online continual learning, each incoming small batch of data is first used for testing and then added to the training set, making the problem truly online. We introduce a new benchmark for online continual visual learning that exhibits large scale and natural distribution shifts.
  arXiv  Detail & Related papers  (2021-08-20T06:17:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.