Sequence Generation Modeling for Continuous Value Prediction

• URL: http://arxiv.org/abs/2412.20211v2
• Date: Fri, 24 Jan 2025 11:18:26 GMT
• Title: Sequence Generation Modeling for Continuous Value Prediction
• Authors: Hongxu Ma, Kai Tian, Tao Zhang, Xuefeng Zhang, Chunjie Chen, Han Li, Jihong Guan, Shuigeng Zhou,
• Abstract summary: Continuous value prediction (CVP) plays a crucial role in short video recommendation, capturing user preferences through precise numerical estimations. We introduce a novel Generative Regression framework for CVP inspired by sequence generation techniques in language modeling. Our method transforms numerical values into token sequences through structural discretization, preserving original data fidelity while improving prediction precision.
• Score: 35.33333441236041
• License:
• Abstract: Continuous value prediction (CVP) plays a crucial role in short video recommendation, capturing user preferences through precise numerical estimations. However, traditional regression-based methods often struggle with challenges like wide value ranges and imbalanced data, leading to prediction bias. While ordinal classification approaches have been introduced to address these issues, their reliance on discretization reduces accuracy and overlooks inherent relationships between intervals. To overcome these limitations, we introduce a novel Generative Regression (GR) framework for CVP, inspired by sequence generation techniques in language modeling. Our method transforms numerical values into token sequences through structural discretization, preserving original data fidelity while improving prediction precision. Leveraging a carefully crafted vocabulary and label encoding, GR employs curriculum learning with an embedding mixup strategy to bridge training-inference gaps. Experimental evaluations on four public datasets and one large-scale industrial dataset validate the superiority of GR over existing methods. Real-world A/B tests on Kuaishou, a leading video platform, further demonstrate its practical effectiveness. Additionally, GR proves adaptable to other regression tasks, such as Lifetime Value (LTV) prediction, showcasing its potential as a robust solution for diverse CVP challenges.

Related papers

• AdaPRL: Adaptive Pairwise Regression Learning with Uncertainty Estimation for Universal Regression Tasks [0.0]
  We propose a novel adaptive pairwise learning framework for regression tasks (AdaPRL) AdaPRL leverages the relative differences between data points and with deep probabilistic models to quantify the uncertainty associated with predictions. Experiments show that AdaPRL can be seamlessly integrated into recently proposed regression frameworks to gain performance improvement.
  arXiv  Detail & Related papers  (2025-01-10T09:19:10Z)
• Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering [55.15192437680943]
  Generative models lack rigorous statistical guarantees for their outputs. We propose a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee. This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example.
  arXiv  Detail & Related papers  (2024-10-02T15:26:52Z)
• A Conditioned Unsupervised Regression Framework Attuned to the Dynamic Nature of Data Streams [0.0]
  This paper presents an optimal strategy for streaming contexts with limited labeled data, introducing an adaptive technique for unsupervised regression. The proposed method leverages a sparse set of initial labels and introduces an innovative drift detection mechanism. To enhance adaptability, we integrate the ADWIN (ADaptive WINdowing) algorithm with error generalization based on Root Mean Square Error (RMSE)
  arXiv  Detail & Related papers  (2023-12-12T19:23:54Z)
• Explainable Parallel RCNN with Novel Feature Representation for Time Series Forecasting [0.0]
  Time series forecasting is a fundamental challenge in data science. We develop a parallel deep learning framework composed of RNN and CNN. Extensive experiments on three datasets reveal the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-05-08T17:20:13Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Cluster Regularization via a Hierarchical Feature Regression [0.0]
  This paper proposes a novel cluster-based regularization - the hierarchical feature regression (HFR) It mobilizes insights from the domains of machine learning and graph theory to estimate parameters along a supervised hierarchical representation of the predictor set. An application to the prediction of economic growth is used to illustrate the HFR's effectiveness in an empirical setting.
  arXiv  Detail & Related papers  (2021-07-10T13:03:01Z)
• Robust Validation: Confident Predictions Even When Distributions Shift [19.327409270934474]
  We describe procedures for robust predictive inference, where a model provides uncertainty estimates on its predictions rather than point predictions. We present a method that produces prediction sets (almost exactly) giving the right coverage level for any test distribution in an $f$-divergence ball around the training population. An essential component of our methodology is to estimate the amount of expected future data shift and build robustness to it.
  arXiv  Detail & Related papers  (2020-08-10T17:09:16Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.