PRIME: Mitigating Subgroup Optimization Competition in Shared CTR Top Networks with Plug-in Residual Input-Conditioned Mixture of Expert

• URL: http://arxiv.org/abs/2608.30449v1
• Date: Mon, 31 Aug 2026 08:37:44 GMT
• Title: PRIME: Mitigating Subgroup Optimization Competition in Shared CTR Top Networks with Plug-in Residual Input-Conditioned Mixture of Expert
• Abstract summary: We study the competition on Avazu with 4 models and 4 semantic fields.<n>We introduce PRIME (Plug-in Residual Input-conditioned Mixture of Experts), a Dense-anchored mixture of low-rank residual experts.
• Score: 16.073701450851008
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Click-through rate (CTR) models vary in feature-interaction design, yet their top networks usually remain a single multilayer perceptron shared by all examples. Heterogeneous user, item, and context subgroups therefore update the same parameters; weakly aligned learning signals make the aggregate gradient a compromise among competing directions. We study the competition on Avazu with 4 models and 4 semantic fields. Across all architectures, semantic subgroups show lower Top-NN gradient cosine similarity than random groups matched by sample size and label ratio, with reductions of 0.23-0.37. This competition motivates input-conditioned experts, but directly replacing an established Dense mapping changes its initial function, sharing pattern, and capacity, obscuring the source of gains. We introduce PRIME (Plug-in Residual Input-conditioned Mixture of Experts), a Dense-anchored mixture of low-rank residual experts. PRIME anchors the original prediction and uses zero-residual initialization to match the Dense baseline exactly at training onset. Input-dependent routing weights low-rank experts for example-specific logit corrections; multi-bag aggregation and EMA load biases stabilize conditional estimation. We evaluate PRIME on held-out Avazu and Criteo test sets across 13 CTR architectures and five paired seeds. Median paired AUC gains are +0.0022 and +0.0066, with LogLoss reductions of 0.0011 and 0.0081, respectively. On FiBiNET and DCNv2, PRIME outperforms APG in all ten seed-level AUC comparisons while using fewer parameters and lower inference latency on both backbones. These results show that function-preserving conditional residuals add input-dependent capacity while preserving the Dense path and its optimization stability. Code is available at https://github.com/YH-learning/PRIME.

Related papers

• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Evolution-Inspired Sample Competition for Deep Neural Network Optimization [19.75458355825581]
  We present textitNatural Selection (NS), a novel evolution-inspired optimization method that explicitly incorporates competitive interactions into deep network training.<n>NS estimates the competitive status of each sample in a group-wise context and uses it to adaptively regulate its training contribution.<n>NS is compatible with diverse network architectures and does not depend on task-specific assumptions.
  arXiv  Detail & Related papers  (2026-04-14T10:48:01Z)
• Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration [76.08899010904652]
  CapCal is a training-free framework that mechanically decouples positional bias from ranking decisions.<n>It achieves superior performance among training-free methods while preserving single-pass efficiency.
  arXiv  Detail & Related papers  (2026-04-11T10:47:22Z)
• FedSQ: Optimized Weight Averaging via Fixed Gating [1.2058208023553034]
  Federated learning enables collaborative training across organizations without sharing raw data.<n>We propose FedSQ (Federated Structural-Quantitative learning), a transfer-d neural federated procedure based on a DualCopy.<n>Fixing the gating reduces learning to within-regime affine refinements, which stabilizes aggregation under heterogeneous partitions.
  arXiv  Detail & Related papers  (2026-04-03T11:54:23Z)
• Do Generative Metrics Predict YOLO Performance? An Evaluation Across Models, Augmentation Ratios, and Dataset Complexity [43.338311770275745]
  We present a controlled evaluation of synthetic augmentation for YOLOv11 across three single-class detection regimes.<n>We benchmark six GAN-, diffusion-, and hybrid-based generators over augmentation ratios from 10% to 150% of the real training split.<n>For each dataset-generator-augmentation configuration, we compute pre-training dataset metrics under a matched-size bootstrap protocol.
  arXiv  Detail & Related papers  (2026-02-20T03:02:36Z)
• Mixing Configurations for Downstream Prediction [20.099477835723704]
  Humans possess an innate ability to group objects by similarity, a cognitive mechanism that clustering algorithms aim to emulate.<n>Recent advances in community detection have enabled the discovery of configurations without requiring labeled data.<n>GraMixC is a plug-and-play module that extracts configurations, aligns them using our Reverse Merge/Split (RMS) technique, and fuses them via attention heads before forwarding them to any downstream predictor.
  arXiv  Detail & Related papers  (2025-10-22T05:09:53Z)
• AEGIS: Authentic Edge Growth In Sparsity for Link Prediction in Edge-Sparse Bipartite Knowledge Graphs [0.8594140167290097]
  AEGIS is an edge-only augmentation framework that resamples existing training edges.<n>On text-rich GDP graph, semantic KNN achieves the largest AUC improvement and Brier score reduction.
  arXiv  Detail & Related papers  (2025-09-26T07:51:40Z)
• AHDMIL: Asymmetric Hierarchical Distillation Multi-Instance Learning for Fast and Accurate Whole-Slide Image Classification [51.525891360380285]
  AHDMIL is an Asymmetric Hierarchical Distillation Multi-Instance Learning framework.<n>It eliminates irrelevant patches through a two-step training process.<n>It consistently outperforms previous state-of-the-art methods in both classification performance and inference speed.
  arXiv  Detail & Related papers  (2025-08-07T07:47:16Z)
• Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness [61.45587642780908]
  We propose a three-step approach for parameter-efficient fine-tuning of image-text foundation models.<n>Our method improves its two key components: minority samples identification and the robust training algorithm.<n>Our theoretical analysis shows that our PPA enhances minority group identification and is Bayes optimal for minimizing the balanced group error.
  arXiv  Detail & Related papers  (2025-03-12T15:46:12Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Semi-supervised Contrastive Learning with Similarity Co-calibration [72.38187308270135]
  We propose a novel training strategy, termed as Semi-supervised Contrastive Learning (SsCL) SsCL combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning. We show that SsCL produces more discriminative representation and is beneficial to few shot learning.
  arXiv  Detail & Related papers  (2021-05-16T09:13:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.