Recurrent Point Review Models
- URL: http://arxiv.org/abs/2012.05684v1
- Date: Thu, 10 Dec 2020 14:11:42 GMT
- Title: Recurrent Point Review Models
- Authors: Kostadin Cvejoski, Ramses J. Sanchez, Bogdan Georgiev, Christian
Bauckhage and Cesar Ojeda
- Abstract summary: We build on deep neural network models to incorporate temporal information and model how to review data changes with time.
We use the dynamic representations of recurrent point process models, which encode the history of how business or service reviews are received in time.
We deploy our methodologies in the context of recommender systems, effectively characterizing the change in preference and taste of users as time evolves.
- Score: 1.412197703754359
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep neural network models represent the state-of-the-art methodologies for
natural language processing. Here we build on top of these methodologies to
incorporate temporal information and model how to review data changes with
time. Specifically, we use the dynamic representations of recurrent point
process models, which encode the history of how business or service reviews are
received in time, to generate instantaneous language models with improved
prediction capabilities. Simultaneously, our methodologies enhance the
predictive power of our point process models by incorporating summarized review
content representations. We provide recurrent network and temporal convolution
solutions for modeling the review content. We deploy our methodologies in the
context of recommender systems, effectively characterizing the change in
preference and taste of users as time evolves. Source code is available at [1].
Related papers
- Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
"Context is Key" (CiK) is a time series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context.
We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters.
Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings.
arXiv Detail & Related papers (2024-10-24T17:56:08Z) - Enforcing Interpretability in Time Series Transformers: A Concept Bottleneck Framework [2.8470354623829577]
We develop a framework based on Concept Bottleneck Models to enforce interpretability of time series Transformers.
We modify the training objective to encourage a model to develop representations similar to predefined interpretable concepts.
We find that the model performance remains mostly unaffected, while the model shows much improved interpretability.
arXiv Detail & Related papers (2024-10-08T14:22:40Z) - Revisiting Dynamic Evaluation: Online Adaptation for Large Language
Models [88.47454470043552]
We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation.
Online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights.
arXiv Detail & Related papers (2024-03-03T14:03:48Z) - QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement.
QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights.
We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
arXiv Detail & Related papers (2023-11-06T00:21:44Z) - Generalization Properties of Retrieval-based Models [50.35325326050263]
Retrieval-based machine learning methods have enjoyed success on a wide range of problems.
Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored.
We present a formal treatment of retrieval-based models to characterize their generalization ability.
arXiv Detail & Related papers (2022-10-06T00:33:01Z) - Dynamic Review-based Recommenders [1.5427245397603195]
We leverage the known power of reviews to enhance rating predictions in a way that respects the causality of review generation.
Our representations are time-interval aware and thus yield a continuous-time representation of the dynamics.
arXiv Detail & Related papers (2021-10-27T20:17:47Z) - How do I update my model? On the resilience of Predictive Process
Monitoring models to change [15.29342790344802]
Predictive Process Monitoring techniques typically construct a predictive model based on past process executions, and then use it to predict the future of new ongoing cases.
This can make Predictive Process Monitoring too rigid to deal with the variability of processes working in real environments.
We evaluate the use of three different strategies that allow the periodic rediscovery or incremental construction of the predictive model.
arXiv Detail & Related papers (2021-09-08T08:50:56Z) - Multi-Branch Deep Radial Basis Function Networks for Facial Emotion
Recognition [80.35852245488043]
We propose a CNN based architecture enhanced with multiple branches formed by radial basis function (RBF) units.
RBF units capture local patterns shared by similar instances using an intermediate representation.
We show it is the incorporation of local information what makes the proposed model competitive.
arXiv Detail & Related papers (2021-09-07T21:05:56Z) - Learning Neural Models for Natural Language Processing in the Face of
Distributional Shift [10.990447273771592]
The dominating NLP paradigm of training a strong neural predictor to perform one task on a specific dataset has led to state-of-the-art performance in a variety of applications.
It builds upon the assumption that the data distribution is stationary, ie. that the data is sampled from a fixed distribution both at training and test time.
This way of training is inconsistent with how we as humans are able to learn from and operate within a constantly changing stream of information.
It is ill-adapted to real-world use cases where the data distribution is expected to shift over the course of a model's lifetime
arXiv Detail & Related papers (2021-09-03T14:29:20Z) - Layer-wise Analysis of a Self-supervised Speech Representation Model [26.727775920272205]
Self-supervised learning approaches have been successful for pre-training speech representation models.
Not much has been studied about the type or extent of information encoded in the pre-trained representations themselves.
arXiv Detail & Related papers (2021-07-10T02:13:25Z) - Meta-learning using privileged information for dynamics [66.32254395574994]
We extend the Neural ODE Process model to use additional information within the Learning Using Privileged Information setting.
We validate our extension with experiments showing improved accuracy and calibration on simulated dynamics tasks.
arXiv Detail & Related papers (2021-04-29T12:18:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.