Improving through Interaction: Searching Behavioral Representation Spaces with CMA-ES-IG

• URL: http://arxiv.org/abs/2603.09011v1
• Date: Mon, 09 Mar 2026 23:00:42 GMT
• Title: Improving through Interaction: Searching Behavioral Representation Spaces with CMA-ES-IG
• Abstract summary: We propose an intuitive technique to learn non-expert users' preferences through rankings of robot behaviors.<n> CMA-ES-IG explicitly incorporates user experience considerations into the preference learning process.<n>It scales more effectively to higher-dimensional preference spaces, maintains computational tractability for high-dimensional problems, and is robust to noisy or inconsistent user feedback.
• Score: 9.51762789049331
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robots that interact with humans must adapt to individual users' preferences to operate effectively in human-centered environments. An intuitive and effective technique to learn non-expert users' preferences is through rankings of robot behaviors, e.g., trajectories, gestures, or voices. Existing techniques primarily focus on generating queries that optimize preference learning outcomes, such as sample efficiency or final preference estimation accuracy. However, the focus on outcome overlooks key user expectations in the process of providing these rankings, which can negatively impact users' adoption of robotic systems. This work proposes the Covariance Matrix Adaptation Evolution Strategies with Information Gain (CMA-ES-IG) algorithm. CMA-ES-IG explicitly incorporates user experience considerations into the preference learning process by suggesting perceptually distinct and informative trajectories for users to rank. We demonstrate these benefits through both simulated studies and real-robot experiments. CMA-ES-IG, compared to state-of-the-art alternatives, (1) scales more effectively to higher-dimensional preference spaces, (2) maintains computational tractability for high-dimensional problems, (3) is robust to noisy or inconsistent user feedback, and (4) is preferred by non-expert users in identifying their preferred robot behaviors. This project's code is available at github.com/interaction-lab/CMA-ES-IG

Related papers

• Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy [5.985777189633703]
  AI copilots represent a new generation of AI-powered systems designed to assist users in complex, context-rich tasks.<n>Central to this personalization is preference optimization: the system's ability to detect, interpret, and align with individual user preferences.<n>This survey examines how user preferences are operationalized in AI copilots.
  arXiv  Detail & Related papers  (2025-05-28T02:52:39Z)
• Search-Based Interaction For Conversation Recommendation via Generative Reward Model Based Simulated User [117.82681846559909]
  Conversational recommendation systems (CRSs) use multi-turn interaction to capture user preferences and provide personalized recommendations.<n>We propose a generative reward model based simulated user, named GRSU, for automatic interaction with CRSs.
  arXiv  Detail & Related papers  (2025-04-29T06:37:30Z)
• Improving User Experience in Preference-Based Optimization of Reward Functions for Assistive Robots [5.523009758632668]
  We show that CMA-ES-IG prioritizes the user's experience of the preference learning process. We show that users find our algorithm more intuitive than previous approaches across both physical and social robot tasks.
  arXiv  Detail & Related papers  (2024-11-17T21:52:58Z)
• Learning Manipulation by Predicting Interaction [85.57297574510507]
  We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction. The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
  arXiv  Detail & Related papers  (2024-06-01T13:28:31Z)
• Real Evaluations Tractability using Continuous Goal-Directed Actions in Smart City Applications [3.1158660854608824]
  Continuous Goal-Directed Actions (CGDA) encodes actions as changes of any feature that can be extracted from the environment. Current strategies involve performing evaluations in a simulation, transferring the final joint trajectory to the actual robot. Two different approaches to reduce the number of evaluations using EA, are proposed and compared.
  arXiv  Detail & Related papers  (2024-02-01T15:38:21Z)
• What Matters to You? Towards Visual Representation Alignment for Robot Learning [81.30964736676103]
  When operating in service of people, robots need to optimize rewards aligned with end-user preferences. We propose Representation-Aligned Preference-based Learning (RAPL), a method for solving the visual representation alignment problem.
  arXiv  Detail & Related papers  (2023-10-11T23:04:07Z)
• Optimizing Algorithms From Pairwise User Preferences [23.87058308494074]
  We introduce SortCMA to optimize algorithm parameter configurations in high dimensions based on pairwise user preferences. We apply this method to tuning a commercial depth sensor without ground truth, and to robot social navigation.
  arXiv  Detail & Related papers  (2023-08-08T20:36:59Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Data-driven Koopman Operators for Model-based Shared Control of Human-Machine Systems [66.65503164312705]
  We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex machines. Both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. We find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.
  arXiv  Detail & Related papers  (2020-06-12T14:14:07Z)
• Mining Implicit Entity Preference from User-Item Interaction Data for Knowledge Graph Completion via Adversarial Learning [82.46332224556257]
  We propose a novel adversarial learning approach by leveraging user interaction data for the Knowledge Graph Completion task. Our generator is isolated from user interaction data, and serves to improve the performance of the discriminator. To discover implicit entity preference of users, we design an elaborate collaborative learning algorithms based on graph neural networks.
  arXiv  Detail & Related papers  (2020-03-28T05:47:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.