Is the User Enjoying the Conversation? A Case Study on the Impact on the Reward Function

• URL: http://arxiv.org/abs/2101.05004v1
• Date: Wed, 13 Jan 2021 11:13:07 GMT
• Title: Is the User Enjoying the Conversation? A Case Study on the Impact on the Reward Function
• Authors: Lina M. Rojas-Barahona
• Abstract summary: We adopt deep neural networks that use distributed semantic representation learning for estimating user satisfaction in conversations. We show that the proposed hierarchical network outperforms state-of-the-art quality estimators. Applying these networks to infer the reward function in a Partial Observable Markov Decision Process yields to a great improvement in the task success rate.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The impact of user satisfaction in policy learning task-oriented dialogue systems has long been a subject of research interest. Most current models for estimating the user satisfaction either (i) treat out-of-context short-texts, such as product reviews, or (ii) rely on turn features instead of on distributed semantic representations. In this work we adopt deep neural networks that use distributed semantic representation learning for estimating the user satisfaction in conversations. We evaluate the impact of modelling context length in these networks. Moreover, we show that the proposed hierarchical network outperforms state-of-the-art quality estimators. Furthermore, we show that applying these networks to infer the reward function in a Partial Observable Markov Decision Process (POMDP) yields to a great improvement in the task success rate.

Related papers

• Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networks [58.469818546042696]
  We study the sample efficiency of OPE with human preference and establish a statistical guarantee for it. By appropriately selecting the size of a ReLU network, we show that one can leverage any low-dimensional manifold structure in the Markov decision process.
  arXiv  Detail & Related papers  (2023-10-16T16:27:06Z)
• Influencer Detection with Dynamic Graph Neural Networks [56.1837101824783]
  We investigate different dynamic Graph Neural Networks (GNNs) configurations for influencer detection. We show that using deep multi-head attention in GNN and encoding temporal attributes significantly improves performance.
  arXiv  Detail & Related papers  (2022-11-15T13:00:25Z)
• Preference Enhanced Social Influence Modeling for Network-Aware Cascade Prediction [59.221668173521884]
  We propose a novel framework to promote cascade size prediction by enhancing the user preference modeling. Our end-to-end method makes the user activating process of information diffusion more adaptive and accurate.
  arXiv  Detail & Related papers  (2022-04-18T09:25:06Z)
• Context Uncertainty in Contextual Bandits with Applications to Recommender Systems [16.597836265345634]
  We propose a new type of recurrent neural networks, dubbed recurrent exploration networks (REN), to jointly perform representation learning and effective exploration in the latent space. Our theoretical analysis shows that REN can preserve the rate-linear suboptimal regret even when there exists uncertainty in the learned representations. Our empirical study demonstrates that REN can achieve satisfactory long-term rewards on both synthetic and real-world recommendation datasets, outperforming state-of-the-art models.
  arXiv  Detail & Related papers  (2022-02-01T23:23:50Z)
• On the role of feedback in visual processing: a predictive coding perspective [0.6193838300896449]
  We consider deep convolutional networks (CNNs) as models of feed-forward visual processing and implement Predictive Coding (PC) dynamics. We find that the network increasingly relies on top-down predictions as the noise level increases. In addition, the accuracy of the network implementing PC dynamics significantly increases over time-steps, compared to its equivalent forward network.
  arXiv  Detail & Related papers  (2021-06-08T10:07:23Z)
• Variational Structured Attention Networks for Deep Visual Representation Learning [49.80498066480928]
  We propose a unified deep framework to jointly learn both spatial attention maps and channel attention in a principled manner. Specifically, we integrate the estimation and the interaction of the attentions within a probabilistic representation learning framework. We implement the inference rules within the neural network, thus allowing for end-to-end learning of the probabilistic and the CNN front-end parameters.
  arXiv  Detail & Related papers  (2021-03-05T07:37:24Z)
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation [51.29486247405601]
  We design a framework to generate counterfactuals for raw data instances with the proposed Attribute-Informed Perturbation (AIP) By utilizing generative models conditioned with different attributes, counterfactuals with desired labels can be obtained effectively and efficiently. Experimental results on real-world texts and images demonstrate the effectiveness, sample quality as well as efficiency of our designed framework.
  arXiv  Detail & Related papers  (2021-01-18T08:37:13Z)
• Unsatisfied Today, Satisfied Tomorrow: a simulation framework for performance evaluation of crowdsourcing-based network monitoring [68.8204255655161]
  We propose an empirical framework tailored to assess the quality of the detection of under-performing cells. The framework simulates both the processes of satisfaction surveys delivery and users satisfaction prediction. We use the simulation framework to test empirically the performance of under-performing sites detection in general scenarios.
  arXiv  Detail & Related papers  (2020-10-30T10:03:48Z)
• Knowing your FATE: Friendship, Action and Temporal Explanations for User Engagement Prediction on Social Apps [40.58156024231199]
  We study a novel problem of explainable user engagement prediction for social network Apps. We design an end-to-end neural framework, FATE, which incorporates three key factors that we identify to influence user engagement. FATE is based on a tensor-based graph neural network (GNN), LSTM and a mixture attention mechanism, which allows for (a) predictive explanations based on learned weights across different feature categories, (b) reduced network complexity, and (c) improved performance in both prediction accuracy and training/inference time.
  arXiv  Detail & Related papers  (2020-06-10T02:59:13Z)
• Improving Interaction Quality Estimation with BiLSTMs and the Impact on Dialogue Policy Learning [0.6538911223040175]
  We propose a novel reward based on user satisfaction estimation. We show that it outperforms all previous estimators while learning temporal dependencies implicitly. We show that applying this model results in higher estimated satisfaction, similar task success rates and a higher robustness to noise.
  arXiv  Detail & Related papers  (2020-01-21T15:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.