A Comparative Study on Reward Models for UI Adaptation with Reinforcement Learning

• URL: http://arxiv.org/abs/2308.13937v2
• Date: Mon, 15 Jan 2024 16:42:29 GMT
• Title: A Comparative Study on Reward Models for UI Adaptation with Reinforcement Learning
• Authors: Daniel Gaspar-Figueiredo, Silvia Abrah\~ao, Marta Fern\'andez-Diego, Emilio Insfran
• Abstract summary: Reinforcement learning can be used to personalise interfaces for each context of use. determining the reward of each adaptation alternative is a challenge in RL for UI adaptation. Recent research has explored the use of reward models to address this challenge, but there is currently no empirical evidence on this type of model.
• Score: 0.6899744489931015
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Adapting the User Interface (UI) of software systems to user requirements and the context of use is challenging. The main difficulty consists of suggesting the right adaptation at the right time in the right place in order to make it valuable for end-users. We believe that recent progress in Machine Learning techniques provides useful ways in which to support adaptation more effectively. In particular, Reinforcement learning (RL) can be used to personalise interfaces for each context of use in order to improve the user experience (UX). However, determining the reward of each adaptation alternative is a challenge in RL for UI adaptation. Recent research has explored the use of reward models to address this challenge, but there is currently no empirical evidence on this type of model. In this paper, we propose a confirmatory study design that aims to investigate the effectiveness of two different approaches for the generation of reward models in the context of UI adaptation using RL: (1) by employing a reward model derived exclusively from predictive Human-Computer Interaction (HCI) models (HCI), and (2) by employing predictive HCI models augmented by Human Feedback (HCI&HF). The controlled experiment will use an AB/BA crossover design with two treatments: HCI and HCI&HF. We shall determine how the manipulation of these two treatments will affect the UX when interacting with adaptive user interfaces (AUI). The UX will be measured in terms of user engagement and user satisfaction, which will be operationalized by means of predictive HCI models and the Questionnaire for User Interaction Satisfaction (QUIS), respectively. By comparing the performance of two reward models in terms of their ability to adapt to user preferences with the purpose of improving the UX, our study contributes to the understanding of how reward modelling can facilitate UI adaptation using RL.

Related papers

• Dual Test-time Training for Out-of-distribution Recommender System [91.15209066874694]
  We propose a novel Dual Test-Time-Training framework for OOD Recommendation, termed DT3OR. In DT3OR, we incorporate a model adaptation mechanism during the test-time phase to carefully update the recommendation model. To the best of our knowledge, this paper is the first work to address OOD recommendation via a test-time-training strategy.
  arXiv  Detail & Related papers  (2024-07-22T13:27:51Z)
• Reinforcement Learning-Based Framework for the Intelligent Adaptation of User Interfaces [0.0]
  Adapting the user interface (UI) of software systems to meet the needs and preferences of users is a complex task. Recent advances in Machine Learning (ML) techniques may provide effective means to support the adaptation process. In this paper, we instantiate a reference framework for Intelligent User Interface Adaptation by using Reinforcement Learning (RL) as the ML component.
  arXiv  Detail & Related papers  (2024-05-15T11:14:33Z)
• Personalized Language Modeling from Personalized Human Feedback [49.344833339240566]
  Reinforcement Learning from Human Feedback (RLHF) is commonly used to fine-tune large language models to better align with human preferences. In this work, we aim to address this problem by developing methods for building personalized language models.
  arXiv  Detail & Related papers  (2024-02-06T04:18:58Z)
• Secrets of RLHF in Large Language Models Part II: Reward Modeling [134.97964938009588]
  We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset. We also introduce contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses.
  arXiv  Detail & Related papers  (2024-01-11T17:56:59Z)
• Learning from Interaction: User Interface Adaptation using Reinforcement Learning [0.0]
  This thesis proposes an RL-based UI adaptation framework that uses physiological data. The framework aims to learn from user interactions and make informed adaptations to improve user experience (UX)
  arXiv  Detail & Related papers  (2023-12-12T12:29:18Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Reward Uncertainty for Exploration in Preference-based Reinforcement Learning [88.34958680436552]
  We present an exploration method specifically for preference-based reinforcement learning algorithms. Our main idea is to design an intrinsic reward by measuring the novelty based on learned reward. Our experiments show that exploration bonus from uncertainty in learned reward improves both feedback- and sample-efficiency of preference-based RL algorithms.
  arXiv  Detail & Related papers  (2022-05-24T23:22:10Z)
• Computational Adaptation of XR Interfaces Through Interaction Simulation [4.6193503399184275]
  We discuss a computational approach to adapt XR interfaces with the goal of improving user experience and performance. Our novel model, applied to menu selection tasks, simulates user interactions by considering both cognitive and motor costs.
  arXiv  Detail & Related papers  (2022-04-19T23:37:07Z)
• Adapting User Interfaces with Model-based Reinforcement Learning [47.469980921522115]
  Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy.
  arXiv  Detail & Related papers  (2021-03-11T17:24:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.