Towards Lifelong Learning Embeddings: An Algorithmic Approach to Dynamically Extend Embeddings

• URL: http://arxiv.org/abs/2408.14118v1
• Date: Mon, 26 Aug 2024 09:06:35 GMT
• Title: Towards Lifelong Learning Embeddings: An Algorithmic Approach to Dynamically Extend Embeddings
• Authors: Miguel Alves Gomes, Philipp Meisen, Tobias Meisen,
• Abstract summary: This paper introduces a modular algorithm that extends embedding input size while preserving learned knowledge. The proposed algorithm also incorporates strategies to mitigate the cold start problem associated with new products.
• Score: 4.680296118462097
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The rapid evolution of technology has transformed business operations and customer interactions worldwide, with personalization emerging as a key opportunity for e-commerce companies to engage customers more effectively. The application of machine learning, particularly that of deep learning models, has gained significant traction due to its ability to rapidly recognize patterns in large datasets, thereby offering numerous possibilities for personalization. These models use embeddings to map discrete information, such as product IDs, into a latent vector space, a method increasingly popular in recent years. However, e-commerce's dynamic nature, characterized by frequent new product introductions, poses challenges for these embeddings, which typically require fixed dimensions and inputs, leading to the need for periodic retraining from scratch. This paper introduces a modular algorithm that extends embedding input size while preserving learned knowledge, addressing the challenges posed by e-commerce's dynamism. The proposed algorithm also incorporates strategies to mitigate the cold start problem associated with new products. The results of initial experiments suggest that this method outperforms traditional embeddings.

Related papers

• Consumer Transactions Simulation through Generative Adversarial Networks [0.07373617024876725]
  This paper presents an innovative application of Generative Adversarial Networks (GANs) to generate synthetic retail transaction data. We diverge from conventional methodologies by integrating SKU data into our GAN architecture and using more sophisticated embedding methods. Preliminary results demonstrate enhanced realism in simulated transactions measured by comparing generated items with real ones.
  arXiv  Detail & Related papers  (2024-08-07T09:45:24Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [49.043599241803825]
  Iterative Contrastive Unlearning (ICU) framework consists of three core components. A Knowledge Unlearning Induction module removes specific knowledge through an unlearning loss. A Contrastive Learning Enhancement module to preserve the model's expressive capabilities against the pure unlearning goal. And an Iterative Unlearning Refinement module that dynamically assess the unlearning extent on specific data pieces and make iterative update.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• State-Space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era [59.279784235147254]
  This survey provides an in-depth summary of the latest approaches that are based on recurrent models for sequential data processing. The emerging picture suggests that there is room for thinking of novel routes, constituted by learning algorithms which depart from the standard Backpropagation Through Time.
  arXiv  Detail & Related papers  (2024-06-13T12:51:22Z)
• On the Resurgence of Recurrent Models for Long Sequences -- Survey and Research Opportunities in the Transformer Era [59.279784235147254]
  This survey is aimed at providing an overview of these trends framed under the unifying umbrella of Recurrence. It emphasizes novel research opportunities that become prominent when abandoning the idea of processing long sequences.
  arXiv  Detail & Related papers  (2024-02-12T23:55:55Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [71.63186089279218]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• Machine Learning and Consumer Data [0.4873362301533825]
  The digital revolution has led to the digitization of human behavior, creating unprecedented opportunities to understand observable actions on an unmatched scale. Emerging phenomena such as crowdfunding and crowdsourcing have further illuminated consumer behavior while also introducing new behavioral patterns. Traditional methods used to analyze consumer data fall short in handling the breadth, precision, and scale of emerging data sources.
  arXiv  Detail & Related papers  (2023-06-25T03:58:15Z)
• Symphony in the Latent Space: Provably Integrating High-dimensional Techniques with Non-linear Machine Learning Models [19.824998167546298]
  This paper revisits building machine learning algorithms that involve interactions between entities. We show that it is possible to decouple the learning of high-dimensional interactions from the learning of non-linear feature interactions.
  arXiv  Detail & Related papers  (2022-12-01T20:18:26Z)
• Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation [102.24108167002252]
  We propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T03:54:11Z)
• INTERN: A New Learning Paradigm Towards General Vision [117.3343347061931]
  We develop a new learning paradigm named INTERN. By learning with supervisory signals from multiple sources in multiple stages, the model being trained will develop strong generalizability. In most cases, our models, adapted with only 10% of the training data in the target domain, outperform the counterparts trained with the full set of data.
  arXiv  Detail & Related papers  (2021-11-16T18:42:50Z)
• Friendship is All we Need: A Multi-graph Embedding Approach for Modeling Customer Behavior [1.181206257787103]
  We propose a multi-graph embedding approach for creating a non-linear representation of customers. We are able to predict users' future behavior with a reasonably high accuracy only by having the information of their friendship network.
  arXiv  Detail & Related papers  (2020-10-06T14:50:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.