Continual Learning with Synthetic Boundary Experience Blending

• URL: http://arxiv.org/abs/2507.23534v1
• Date: Thu, 31 Jul 2025 13:20:17 GMT
• Title: Continual Learning with Synthetic Boundary Experience Blending
• Authors: Chih-Fan Hsu, Ming-Ching Chang, Wei-Chao Chen,
• Abstract summary: We propose a novel training framework, bf Experience Blending, which integrates knowledge from both stored key samples and synthetic, boundary-adjacent data.<n>Our method outperforms nine CL baselines, achieving accuracy improvements of 10%, 6%, and 13%, respectively.
• Score: 12.65383500988952
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning (CL) aims to address catastrophic forgetting in models trained sequentially on multiple tasks. While experience replay has shown promise, its effectiveness is often limited by the sparse distribution of stored key samples, leading to overly simplified decision boundaries. We hypothesize that introducing synthetic data near the decision boundary (Synthetic Boundary Data, or SBD) during training serves as an implicit regularizer, improving boundary stability and mitigating forgetting. To validate this hypothesis, we propose a novel training framework, {\bf Experience Blending}, which integrates knowledge from both stored key samples and synthetic, boundary-adjacent data. Experience blending consists of two core components: (1) a multivariate Differential Privacy (DP) noise mechanism that injects batch-wise noise into low-dimensional feature representations, generating SBD; and (2) an end-to-end training strategy that jointly leverages both stored key samples and SBD. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet demonstrate that our method outperforms nine CL baselines, achieving accuracy improvements of 10%, 6%, and 13%, respectively.

Related papers

• Dual-End Consistency Model [41.982957134224904]
  Slow iterative sampling is a major bottleneck for the practical deployment of diffusion and flow-based generative models.<n>We propose a Dual-End Consistency Model (DE-CM) that selects vital sub-trajectory clusters to achieve stable and effective training.<n>Our method achieves a state-of-the-art FID score of 1.70 in one-step generation on the ImageNet 256x256 dataset, outperforming existing CM-based one-step approaches.
  arXiv  Detail & Related papers  (2026-02-11T11:51:01Z)
• Consistency-Regularized GAN for Few-Shot SAR Target Recognition [40.2533418376231]
  Few-shot recognition in synthetic aperture radar (SAR) imagery remains a critical bottleneck for real-world applications due to extreme data scarcity.<n>A promising strategy involves a large dataset with a generative adversarial network (GAN), pre-training a model via self-supervised learning (SSL), and then fine-tuning on the few labeled samples.<n>This approach faces a fundamental paradox: conventional GANs themselves require abundant data for stable training, contradicting the premise of few-shot learning.<n>We propose the consistency-regularized generative adversarial network (Cr-GAN), a novel framework designed to synthesize diverse, high
  arXiv  Detail & Related papers  (2026-01-22T06:02:39Z)
• Modest-Align: Data-Efficient Alignment for Vision-Language Models [67.48633659305592]
  Cross-modal alignment models often suffer from overconfidence and degraded performance when operating in resource-constrained settings.<n>We propose Modest-Align, a lightweight alignment framework designed for robustness and efficiency.<n>Our method offers a practical and scalable solution for cross-modal alignment in real-world, low-resource scenarios.
  arXiv  Detail & Related papers  (2025-10-24T16:11:10Z)
• Decoupled Contrastive Learning for Federated Learning [0.0]
  We introduce Decoupled Contrastive Learning for Federated Learning (DCFL)<n>DCFL is a novel framework that decouples the existing contrastive loss into two objectives.<n>Our results show that DCFL achieves stronger alignment between positive samples and greater uniformity between negative samples.
  arXiv  Detail & Related papers  (2025-08-06T01:39:54Z)
• Variational Supervised Contrastive Learning [50.79938854370321]
  We propose Variational Supervised Contrastive Learning (VarCon), which reformulates supervised contrastive learning as variational inference over latent class variables.<n>VarCon achieves state-of-the-art performance for contrastive learning frameworks, reaching 79.36% Top-1 accuracy on ImageNet-1K and 78.29% on CIFAR-100 with a ResNet-50 encoder.
  arXiv  Detail & Related papers  (2025-06-09T04:19:12Z)
• Offline Learning for Combinatorial Multi-armed Bandits [56.96242764723241]
  Off-CMAB is the first offline learning framework for CMAB.<n>Off-CMAB combines pessimistic reward estimations with solvers.<n>Experiments on synthetic and real-world datasets highlight the superior performance of CLCB.
  arXiv  Detail & Related papers  (2025-01-31T16:56:18Z)
• CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs [6.456189487006878]
  We present CLAMP-ViT, a data-free post-training quantization method for vision transformers (ViTs) We identify the limitations of recent techniques, notably their inability to leverage meaningful inter-patch relationships. CLAMP-ViT employs a two-stage approach, cyclically adapting between data generation and model quantization.
  arXiv  Detail & Related papers  (2024-07-07T05:39:25Z)
• Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection [9.241737058291823]
  Adrial training methods generate independent initial perturbation for adversarial samples from a simple uniform distribution. We propose a simple yet effective training framework called Batch-in-Batch to enhance models. We show that models trained within the BB framework consistently have higher adversarial accuracy across various adversarial settings.
  arXiv  Detail & Related papers  (2024-06-06T13:34:43Z)
• Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching [53.05954114863596]
  We propose a brand-new Deep Boosting Learning (DBL) algorithm for image-text matching. An anchor branch is first trained to provide insights into the data properties. A target branch is concurrently tasked with more adaptive margin constraints to further enlarge the relative distance between matched and unmatched samples.
  arXiv  Detail & Related papers  (2024-04-28T08:44:28Z)
• CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective [48.99488315273868]
  We propose a contrastive knowledge distillation framework that achieves sample-wise logit alignment while preserving semantic consistency.<n>Our approach transfers "dark knowledge" through teacher-student contrastive alignment at the sample level.<n>We conduct comprehensive experiments across three benchmark datasets, including the CIFAR-100, ImageNet-1K, and MS COCO datasets.
  arXiv  Detail & Related papers  (2024-04-22T11:52:40Z)
• Certified Machine Unlearning via Noisy Stochastic Gradient Descent [20.546589699647416]
  Machine unlearning aims to efficiently remove the effect of certain data points on the trained model.<n>We propose to leverage noisy gradient descent for unlearning and establish its first approximate unlearning guarantee.
  arXiv  Detail & Related papers  (2024-03-25T18:43:58Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning [50.809769498312434]
  We propose a novel dataset pruning method termed as Temporal Dual-Depth Scoring (TDDS) Our method achieves 54.51% accuracy with only 10% training data, surpassing random selection by 7.83% and other comparison methods by at least 12.69%.
  arXiv  Detail & Related papers  (2023-11-22T03:45:30Z)
• Towards Sustainable Learning: Coresets for Data-efficient Deep Learning [9.51481812606879]
  CREST is the first scalable subsetx deep networks framework with rigorous theoretical subsetx experiments on datasets. CREST identifies the most valuable examples of a non-Image function.
  arXiv  Detail & Related papers  (2023-06-02T02:51:08Z)
• Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking [83.40120598637665]
  Zero-shot transfer learning for Dialogue State Tracking (DST) helps to handle a variety of task-oriented dialogue domains without the cost of collecting in-domain data. Existing works mainly study common data- or model-level augmentation methods to enhance the generalization. We present a simple and effective "divide, conquer and combine" solution, which explicitly disentangles the semantics of seen data.
  arXiv  Detail & Related papers  (2023-06-01T08:21:20Z)
• Scale-Equivalent Distillation for Semi-Supervised Object Detection [57.59525453301374]
  Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. We analyze the challenges these methods meet with the empirical experiment results. We introduce a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance.
  arXiv  Detail & Related papers  (2022-03-23T07:33:37Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Unpaired Adversarial Learning for Single Image Deraining with Rain-Space Contrastive Constraints [61.40893559933964]
  We develop an effective unpaired SID method which explores mutual properties of the unpaired exemplars by a contrastive learning manner in a GAN framework, named as CDR-GAN. Our method performs favorably against existing unpaired deraining approaches on both synthetic and real-world datasets, even outperforms several fully-supervised or semi-supervised models.
  arXiv  Detail & Related papers  (2021-09-07T10:00:45Z)
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
  We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
  arXiv  Detail & Related papers  (2021-03-24T07:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.