Repeated Padding+: Simple yet Effective Data Augmentation Plugin for Sequential Recommendation

• URL: http://arxiv.org/abs/2403.06372v3
• Date: Tue, 15 Jul 2025 02:45:05 GMT
• Title: Repeated Padding+: Simple yet Effective Data Augmentation Plugin for Sequential Recommendation
• Authors: Yizhou Dang, Yuting Liu, Enneng Yang, Guibing Guo, Linying Jiang, Jianzhe Zhao, Xingwei Wang,
• Abstract summary: We propose a simple yet effective padding method called Repeated Padding+ (RepPad+)<n>Our method contains no trainable parameters or hypersequences and is a plug-and-play data augmentation operation.<n>The average recommendation performance improvement is up to 84.11% on GRU4Rec and 35.34% on SASRec.
• Score: 9.913317029557588
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sequential recommendation aims to provide users with personalized suggestions based on their historical interactions. When training sequential models, padding is a widely adopted technique for two main reasons: 1) The vast majority of models can only handle fixed-length sequences; 2) Batching-based training needs to ensure that the sequences in each batch have the same length. The special value \emph{0} is usually used as the padding content, which does not contain the actual information and is ignored in the model calculations. This common-sense padding strategy leads us to a problem that has never been explored before: Can we fully utilize this idle input space by padding other content to further improve model performance and training efficiency? In this work, we propose a simple yet effective padding method called Repeated Padding+ (RepPad+). Specifically, we use the original interaction sequences as the padding content and fill it to the padding positions during model training. This operation can be performed a finite number of times or repeated until the input sequences' length reaches the maximum limit. For those sequences that can not pad full original data, we draw inspiration from the Sliding Windows strategy and intercept consecutive subsequences to fill in the idle space. Our RepPad+ can be viewed as a sequence-level data augmentation strategy. Unlike most existing works, our method contains no trainable parameters or hyperparameters and is a plug-and-play data augmentation operation. Extensive experiments on various categories of sequential models and seven real-world datasets demonstrate the effectiveness and efficiency of our approach. The average recommendation performance improvement is up to 84.11% on GRU4Rec and 35.34% on SASRec. We also provide in-depth analysis and explanation of what makes RepPad+ effective from multiple perspectives.

Related papers

• Improving Continual Pre-training Through Seamless Data Packing [34.13195340154738]
  We propose a novel data packing strategy aimed at preserving contextual information more effectively and enhancing model performance.<n>Our approach employs a sliding window technique in the first stage that synchronizes overlapping tokens across consecutive sequences.<n>In the second stage, we adopt a First-Fit-Decreasing algorithm to pack shorter texts into bins slightly larger than the target sequence length, thereby minimizing padding and truncation.
  arXiv  Detail & Related papers  (2025-05-28T06:30:37Z)
• Augmenting Sequential Recommendation with Balanced Relevance and Diversity [17.542273338911553]
  We propose a novel Balanced Data Augmentation for Sequential Recommendation (BASRec) to generate data that balance relevance and diversity.<n>BASRec consists of two modules: Single-sequence Augmentation and Cross-sequence Augmentation.<n>We demonstrate that BASRec generates data with a better balance between relevance and diversity than existing methods.
  arXiv  Detail & Related papers  (2024-12-11T11:29:15Z)
• Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation [69.60321475454843]
  We propose DPCPL, the first pre-training and prompt-tuning paradigm tailored for Multi-Behavior Sequential Recommendation. In the pre-training stage, we propose a novel Efficient Behavior Miner (EBM) to filter out the noise at multiple time scales. Subsequently, we propose to tune the pre-trained model in a highly efficient manner with the proposed Customized Prompt Learning (CPL) module.
  arXiv  Detail & Related papers  (2024-08-21T06:48:38Z)
• Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs.<n>Specifically, we introduce a Ghost Spatial Masking (GSM) module, embedded within a Transformer encoder, for spatial feature extraction.<n>We benchmark three practical sports datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)
• Single-Model and Any-Modality for Video Object Tracking [85.83753760853142]
  We introduce Un-Track, a Unified Tracker of a single set of parameters for any modality. To handle any modality, our method learns their common latent space through low-rank factorization and reconstruction techniques. Our Un-Track achieves +8.1 absolute F-score gain, on the DepthTrack dataset, by introducing only +2.14 (over 21.50) GFLOPs with +6.6M (over 93M) parameters.
  arXiv  Detail & Related papers  (2023-11-27T14:17:41Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)
• UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory [69.33445217944029]
  PETL is an effective strategy for adapting pre-trained models to downstream domains. Recent PETL works focus on the more valuable memory-efficient characteristic. We propose a new memory-efficient PETL strategy, Universal Parallel Tuning (UniPT)
  arXiv  Detail & Related papers  (2023-08-28T05:38:43Z)
• Improved Distribution Matching for Dataset Condensation [91.55972945798531]
  We propose a novel dataset condensation method based on distribution matching. Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
  arXiv  Detail & Related papers  (2023-07-19T04:07:33Z)
• Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation [16.00020821220671]
  Sequential recommendation is an important task to predict the next-item to access based on a sequence of items. Most existing works learn user preference as the transition pattern from the previous item to the next one, ignoring the time interval between these two items. We propose to augment sequence data from the perspective of time interval, which is not studied in the literature.
  arXiv  Detail & Related papers  (2022-12-16T03:13:43Z)
• Cross-Modal Adapter for Text-Video Retrieval [91.9575196703281]
  We present a novel $textbfCross-Modal Adapter$ for parameter-efficient fine-tuning. Inspired by adapter-based methods, we adjust the pre-trained model with a few parameterization layers. It achieves superior or comparable performance compared to fully fine-tuned methods on MSR-VTT, MSVD, VATEX, ActivityNet, and DiDeMo datasets.
  arXiv  Detail & Related papers  (2022-11-17T16:15:30Z)
• Pre-training helps Bayesian optimization too [49.28382118032923]
  We seek an alternative practice for setting functional priors. In particular, we consider the scenario where we have data from similar functions that allow us to pre-train a tighter distribution a priori. Our results show that our method is able to locate good hyper parameters at least 3 times more efficiently than the best competing methods.
  arXiv  Detail & Related papers  (2022-07-07T04:42:54Z)
• Sequence Adaptation via Reinforcement Learning in Recommender Systems [8.909115457491522]
  We propose the SAR model, which learns the sequential patterns and adjusts the sequence length of user-item interactions in a personalized manner. In addition, we optimize a joint loss function to align the accuracy of the sequential recommendations with the expected cumulative rewards of the critic network. Our experimental evaluation on four real-world datasets demonstrates the superiority of our proposed model over several baseline approaches.
  arXiv  Detail & Related papers  (2021-07-31T13:56:46Z)
• Pretraining Representations for Data-Efficient Reinforcement Learning [12.43475487724972]
  We use unlabeled data to pretrain an encoder which is then finetuned on a small amount of task-specific data. When limited to 100k steps of interaction on Atari games, our approach significantly surpasses prior work. Our approach shows particular promise when combined with larger models as well as more diverse, task-aligned observational data.
  arXiv  Detail & Related papers  (2021-06-09T04:14:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.