FSE: Continual Learning for Named Entity Recognition by Fast-Slow Experts

• URL: http://arxiv.org/abs/2607.22075v1
• Date: Fri, 24 Jul 2026 08:20:01 GMT
• Title: FSE: Continual Learning for Named Entity Recognition by Fast-Slow Experts
• Abstract summary: This paper proposes Fast-Slow Experts enhanced span-based NER model for CLNER.<n>The shared fast expert learns token-level links to efficiently filter out unlikely spans, while the task-specific slow expert performs span classification only on the remaining candidates.<n>It stabilizes learning by promoting knowledge sharing across tasks and maintains plasticity by reducing learning burden at each task.
• Score: 16.32828603043781
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual Learning for Named Entity Recognition (CLNER) enable models to incrementally learn new entity types without forgetting previously acquired ones. However, existing methods suffer from catastrophic forgetting and insufficient exploitation of shared information across tasks. This paper proposes FSE, a Fast-Slow Experts enhanced span-based NER model for CLNER. The shared fast expert learns token-level links to efficiently filter out unlikely spans, while the task-specific slow expert performs span classification only on the remaining candidates. It stabilizes learning by promoting knowledge sharing across tasks and maintains plasticity by reducing learning burden at each task. A length-decay negative sampling strategy to mitigate span imbalance is also introduced. Extensive experiments on OntoNotes and FewNERD synthestic datasets demonstrate that FSE achieves state-of-the-art performance in CLNER scenarios, with effectiveness of each component, empirical evidence of faster convergence and expected functionality of both experts.

Related papers

• Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition [52.07624601400713]
  In-context learning (ICL) with large language models (LLMs) has emerged as a powerful alternative to fine-tuning for Named Entity Recognition (NER)<n>We conduct a comprehensive investigation of many-shot ICL for NER and explore its effectiveness in annotating and refining data for low-resource NER tasks.<n>Our experiments demonstrate that: (1) scaling to hundreds of in-context examples enables LLMs to match or even surpass the performance of fully supervised BERT models; and (2) using about one hundred human-labeled examples as demonstrations, many-shot in-context annotation can generate high-quality labeled
  arXiv  Detail & Related papers  (2026-06-20T05:39:59Z)
• Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning [10.01449025634975]
  Existing methods treat parameters uniformly, failing to distinguish between specific task knowledge and shared capabilities.<n>We introduce Mixture of Sparse Experts for Task Agnostic Continual Learning (SETA), a framework that resolves the plasticity-stability conflict through adaptive sparse subspace decomposition.<n>We show SETA achieves competitive or superior overall performance relative to state-of-the-art continual learning baselines.
  arXiv  Detail & Related papers  (2026-06-05T17:53:52Z)
• Split-on-Share: Mixture of Sparse Experts for Task-Agnostic Continual Learning [10.01449025634975]
  Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma.<n>We introduce SETA, a framework that resolves the plasticity-stability conflict by decomposing the model into modular subspaces.<n>We show that SETA consistently outperforms state-of-the-art parameter-efficient fine-tuning-based continual learning methods.
  arXiv  Detail & Related papers  (2026-01-24T22:39:22Z)
• One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning [52.966712416640085]
  We propose SMoPE, a novel framework that integrates the benefits of both task-specific and shared prompt strategies.<n>SMoPE consistently outperforms task-specific prompt methods and achieves performance competitive with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-09-29T08:54:58Z)
• SemiOccam: A Robust Semi-Supervised Image Recognition Network Using Sparse Labels [3.37079635422936]
  SemiOccam is an image recognition network that leverages semi-supervised learning in a highly efficient manner.<n>We construct a hierarchical mixture density classification mechanism by optimizing mutual information between feature representations and target classes.<n>Our method achieves state-of-the-art performance on three commonly used datasets.
  arXiv  Detail & Related papers  (2025-06-04T05:24:28Z)
• CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition [3.695767900907561]
  CLLMFS is a Contrastive Learning enhanced Large Language Model framework for Few-Shot Named Entity Recognition. It integrates Low-Rank Adaptation (LoRA) and contrastive learning mechanisms specifically tailored for few-shot NER. Our method has achieved state-of-the-art performance improvements on F1-score ranging from 2.58% to 97.74% over existing best-performing methods.
  arXiv  Detail & Related papers  (2024-08-23T04:44:05Z)
• FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models [50.331708897857574]
  We introduce FactorLLM, a novel approach that decomposes well-trained dense FFNs into sparse sub-networks without requiring any further modifications. FactorLLM achieves comparable performance to the source model securing up to 85% model performance while obtaining over a 30% increase in inference speed.
  arXiv  Detail & Related papers  (2024-08-15T16:45:16Z)
• Relaxed Contrastive Learning for Federated Learning [48.96253206661268]
  We propose a novel contrastive learning framework to address the challenges of data heterogeneity in federated learning. Our framework outperforms all existing federated learning approaches by huge margins on the standard benchmarks.
  arXiv  Detail & Related papers  (2024-01-10T04:55:24Z)
• Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality [55.88910947643436]
  Self-supervised pre-training is essential for handling vast quantities of unlabeled data in practice. HiDe-Prompt is an innovative approach that explicitly optimize the hierarchical components with an ensemble of task-specific prompts and statistics. Our experiments demonstrate the superior performance of HiDe-Prompt and its robustness to pre-training paradigms in continual learning.
  arXiv  Detail & Related papers  (2023-10-11T06:51:46Z)
• Nested Collaborative Learning for Long-Tailed Visual Recognition [71.6074806468641]
  NCL consists of two core components, namely Nested Individual Learning (NIL) and Nested Balanced Online Distillation (NBOD) To learn representations more thoroughly, both NIL and NBOD are formulated in a nested way, in which the learning is conducted on not just all categories from a full perspective but some hard categories from a partial perspective. In the NCL, the learning from two perspectives is nested, highly related and complementary, and helps the network to capture not only global and robust features but also meticulous distinguishing ability.
  arXiv  Detail & Related papers  (2022-03-29T08:55:39Z)
• Low-Regret Active learning [64.36270166907788]
  We develop an online learning algorithm for identifying unlabeled data points that are most informative for training. At the core of our work is an efficient algorithm for sleeping experts that is tailored to achieve low regret on predictable (easy) instances.
  arXiv  Detail & Related papers  (2021-04-06T22:53:45Z)
• Learn-Prune-Share for Lifelong Learning [25.678753894026357]
  We propose a learn-prune-share (LPS) algorithm which addresses the challenges of catastrophic forgetting, parsimony, and knowledge reuse simultaneously. LPS splits the network into task-specific partitions via an ADMM-based pruning strategy. This leads to no forgetting, while maintaining parsimony.
  arXiv  Detail & Related papers  (2020-12-13T04:05:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.