Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edge

• URL: http://arxiv.org/abs/2604.07399v1
• Date: Wed, 08 Apr 2026 09:16:21 GMT
• Title: Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edge
• Abstract summary: Continual learning on edge devices requires high accuracy and training-time efficiency to support on-device adaptation.<n>We propose CPS-Prompt, a patch-aware sparse prompting framework that explicitly targets training-time memory usage and computational cost.<n> Experiments show that CPS-Prompt improves peak memory, training time, and energy efficiency by about 1.6x over the balanced CODA-Prompt baseline.
• Score: 6.483776204899318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning (CL) on edge devices requires not only high accuracy but also training-time efficiency to support on-device adaptation under strict memory and computational constraints. While prompt-based continual learning (PCL) is parameter-efficient and achieves competitive accuracy, prior work has focused mainly on accuracy or inference-time performance, often overlooking the memory and computational costs of on-device training. In this paper, we propose CPS-Prompt, a critical patch-aware sparse prompting framework that explicitly targets training-time memory usage and computational cost by integrating critical patch sampling (CPS) for task-aware token reduction and decoupled prompt and classifier training (DPCT) to reduce backpropagation overhead. Experiments on three public benchmarks and real edge hardware show that CPS-Prompt improves peak memory, training time, and energy efficiency by about 1.6x over the balanced CODA-Prompt baseline, while maintaining accuracy within 2% of the state-of-the-art C-Prompt on average and remaining competitive with CODA-Prompt in accuracy. The code is available at https://github.com/laymond1/cps-prompt.

Related papers

• CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration [67.17709877327232]
  Cate KV is a hybrid KV cache method that retains only critical token information for consistent heads.<n>Cate KV reduces memory usage by up to $2.72times$ and accelerates decoding by $2.18times$ in single-sample inputs.
  arXiv  Detail & Related papers  (2026-08-31T06:02:37Z)
• Training-Free Hashing-Based Attention via Binary Principal Components [62.88957443238231]
  Long-context large language models (LLMs) are increasingly deployed in real-world applications.<n>Existing sparse attention reduce computation by attending to fewer KV pairs, but often suffer from substantial accuracy degradation.<n>We present BinaryPC, a training-free, data-aware hashing-based sparse attention for long-context LLMs.
  arXiv  Detail & Related papers  (2026-08-05T03:19:21Z)
• CSAttention: Centroid-Scoring Attention for Accelerating LLM Inference [3.1255988998610307]
  Centroid-Scoring Attention (CSAttention) is a training-free sparse attention method optimized for high- throughput contexts.<n>It front-loads computation into a one-time offline prefill phase that can be amortized across multiple queries.<n>It consistently outperforms state-of-the-art sparse attention methods in both model accuracy and inference speed.
  arXiv  Detail & Related papers  (2026-03-30T01:42:29Z)
• Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
  Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
  arXiv  Detail & Related papers  (2026-02-13T02:13:45Z)
• FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness [5.2612663135589175]
  Scaling training compute, measured in FLOPs, has long been shown to improve the accuracy of large language models.<n>We introduce TTC-aware training, where an intermediate checkpoint and a corresponding TTC configuration can together match or exceed the accuracy of a fully trained model.<n>Building on this insight, we propose an early stopping algorithm that jointly selects a checkpoint and TTC configuration to minimize training compute without sacrificing accuracy.
  arXiv  Detail & Related papers  (2026-01-04T02:33:30Z)
• Training-free Context-adaptive Attention for Efficient Long Context Modeling [57.703159205740185]
  Training-free Context-adaptive Attention (TCA-Attention) is a training-free sparse attention mechanism that selectively attends to only the informative tokens for efficient long-context inference.<n>TCA-Attention achieves a 2.8$times$ speedup and reduces KV cache by 61% at 128K context length while maintaining performance comparable to full attention.
  arXiv  Detail & Related papers  (2025-12-10T01:54:57Z)
• Efficient Long-context Language Model Training by Core Attention Disaggregation [40.14172357304901]
  We present a technique that improves long-context large language model training by decoupling the core attention computation, softmax(QKT)V, from the rest of the model.<n>We implement CAD in a system called DistCA, which uses a ping-pong execution scheme to fully overlap communication with computation and in-place execution on attention servers to reduce memory use.
  arXiv  Detail & Related papers  (2025-10-20T21:40:51Z)
• Training Long-Context LLMs Efficiently via Chunk-wise Optimization [60.05884946552877]
  We present textitSequential Chunk-wise Optimization (SeCO), a memory-efficient training paradigm that partitions lengthy inputs into manageable chunks.<n>We also introduce textitSparse Chunk-wise Optimization (SpaCO), which reduces computational overhead by selectively propagating gradients to specific chunks.<n>SpaCO decouples the computational cost of backpropagation from the context length, enabling training time to gradually converge to inference time as sequences become longer.
  arXiv  Detail & Related papers  (2025-05-22T14:11:34Z)
• DPQ-HD: Post-Training Compression for Ultra-Low Power Hyperdimensional Computing [6.378578005171813]
  We propose a novel Post Training Compression algorithm, Decomposition-Pruning-Quantization (DPQ-HD)<n>DPQ-HD reduces computational and memory overhead by uniquely combining the above three compression techniques.<n>We demonstrate that DPQ-HD achieves up to 20-100x reduction in memory for image and graph classification tasks with only a 1-2% drop in accuracy.
  arXiv  Detail & Related papers  (2025-05-08T16:54:48Z)
• Failure Tolerant Training with Persistent Memory Disaggregation over CXL [7.700500756012469]
  This paper proposes TRAININGCXL that can efficiently process large-scale recommendation datasets in the pool of disaggregated memory. To this end, we integrate persistent memory (PMEM) and GPU into a cache-coherent domain as Type-2. The evaluation shows that TRAININGCXL achieves 5.2x training performance improvement and 76% energy savings, compared to the modern PMEM-based recommendation systems.
  arXiv  Detail & Related papers  (2023-01-14T05:59:07Z)
• DIVISION: Memory Efficient Training via Dual Activation Precision [60.153754740511864]
  State-of-the-art work combines a search of quantization bit-width with the training, which makes the procedure complicated and less transparent. We propose a simple and effective method to compress DNN training. Experiment results show DIVISION has better comprehensive performance than state-of-the-art methods, including over 10x compression of activation maps and competitive training throughput, without loss of model accuracy.
  arXiv  Detail & Related papers  (2022-08-05T03:15:28Z)
• On-Device Training Under 256KB Memory [62.95579393237751]
  We propose an algorithm-system co-design framework to make on-device training possible with only 256KB of memory. Our framework is the first solution to enable tiny on-device training of convolutional neural networks under 256KB and 1MB Flash.
  arXiv  Detail & Related papers  (2022-06-30T17:59:08Z)
• Online Convolutional Re-parameterization [51.97831675242173]
  We present online convolutional re- parameterization (OREPA), a two-stage pipeline, aiming to reduce the huge training overhead by squeezing the complex training-time block into a single convolution. Compared with the state-of-the-art re-param models, OREPA is able to save the training-time memory cost by about 70% and accelerate the training speed by around 2x. We also conduct experiments on object detection and semantic segmentation and show consistent improvements on the downstream tasks.
  arXiv  Detail & Related papers  (2022-04-02T09:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.