ADAPT: Actively Discovering and Adapting to Preferences for any Task

• URL: http://arxiv.org/abs/2504.04040v1
• Date: Sat, 05 Apr 2025 03:16:22 GMT
• Title: ADAPT: Actively Discovering and Adapting to Preferences for any Task
• Abstract summary: We introduce a benchmark designed to evaluate agents' ability to adhere to user preferences across various household tasks through active questioning.<n>Next, we propose Reflection-DPO, a novel training approach for adapting large language models to the task of active questioning.<n>We find that prior approaches that use state-of-the-art LLMs fail to sufficiently follow user preferences in ADAPT due to insufficient questioning and poor adherence to elicited preferences.
• Score: 34.92667747380468
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Assistive agents should be able to perform under-specified long-horizon tasks while respecting user preferences. We introduce Actively Discovering and Adapting to Preferences for any Task (ADAPT) -- a benchmark designed to evaluate agents' ability to adhere to user preferences across various household tasks through active questioning. Next, we propose Reflection-DPO, a novel training approach for adapting large language models (LLMs) to the task of active questioning. Reflection-DPO finetunes a 'student' LLM to follow the actions of a privileged 'teacher' LLM, and optionally ask a question to gather necessary information to better predict the teacher action. We find that prior approaches that use state-of-the-art LLMs fail to sufficiently follow user preferences in ADAPT due to insufficient questioning and poor adherence to elicited preferences. In contrast, Reflection-DPO achieves a higher rate of satisfying user preferences, outperforming a zero-shot chain-of-thought baseline by 6.1% on unseen users.

Related papers

• Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization [61.641777037967366]
  Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns.<n>Agentic reinforcement learning (RL) has emerged as a promising solution for training such agents in multi-turn settings.<n>We propose BAO, an agentic RL framework that combines behavior enhancement to enrich proactive reasoning and information-gathering capabilities.
  arXiv  Detail & Related papers  (2026-02-11T20:40:43Z)
• CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions [39.554239954719876]
  CUPID is a benchmark of 756 human-curated interaction session histories.<n>We evaluate 10 open and proprietary Large Language Models (LLMs)<n>Our work highlights the need to advance LLM capabilities for more contextually personalized interactions.
  arXiv  Detail & Related papers  (2025-08-03T09:04:48Z)
• Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent [56.61028117645315]
  We propose a novel thought-augmented interactive recommender agent system (TAIRA) that addresses complex user intents through distilled thought patterns.<n>Specifically, TAIRA is designed as an LLM-powered multi-agent system featuring a manager agent that orchestrates recommendation tasks by decomposing user needs and planning subtasks.<n>Through comprehensive experiments conducted across multiple datasets, TAIRA exhibits significantly enhanced performance compared to existing methods.
  arXiv  Detail & Related papers  (2025-06-30T03:15:50Z)
• UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering [39.79275025010785]
  name is a benchmark designed to evaluate the effectiveness of user embeddings in prompting large language models for personalization.<n>We conduct extensive experiments on various state-of-the-art methods for modeling user embeddings.
  arXiv  Detail & Related papers  (2025-02-26T14:34:00Z)
• Reason4Rec: Large Language Models for Recommendation with Deliberative User Preference Alignment [69.11529841118671]
  We propose a new Deliberative Recommendation task, which incorporates explicit reasoning about user preferences as an additional alignment goal.<n>We then introduce the Reasoning-powered Recommender framework for deliberative user preference alignment.
  arXiv  Detail & Related papers  (2025-02-04T07:17:54Z)
• Reinforced Prompt Personalization for Recommendation with Large Language Models [24.360796133889156]
  We introduce the concept of instance-wise prompting, aiming at personalizing discrete prompts for individual users.<n>To improve efficiency and quality, RPP personalizes prompts at the sentence level rather than searching in the vast vocabulary word-by-word.
  arXiv  Detail & Related papers  (2024-07-24T09:24:49Z)
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization [73.7779735046424]
  We show that different prompts should be adapted to different Large Language Models (LLM) to enhance their capabilities across various downstream tasks in NLP. We then propose a model-adaptive prompt (MAPO) method that optimize the original prompts for each specific LLM in downstream tasks.
  arXiv  Detail & Related papers  (2024-07-04T18:39:59Z)
• MACAROON: Training Vision-Language Models To Be Your Engaged Partners [95.32771929749514]
  Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unlabeled. In this study, we aim to shift LVLMs from passive answer providers to proactive engaged partners. We introduce MACAROON, self-iMaginAtion for ContrAstive pReference OptimizatiON, which instructs LVLMs to autonomously generate contrastive response pairs for unlabeled questions.
  arXiv  Detail & Related papers  (2024-06-20T09:27:33Z)
• Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization [19.200989737492595]
  Large language models (LLMs) have shown great progress in responding to user questions. The quality of LLM outputs heavily depends on the prompt design, where a good prompt might enable the LLM to answer a very challenging question correctly. We propose a hierarchy of LLMs, first constructing a prompt with precise instructions and accurate wording in a hierarchical manner, and then using this prompt to generate the final answer to the user query.
  arXiv  Detail & Related papers  (2024-05-30T17:05:45Z)
• Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts [95.09994361995389]
  Relative Preference Optimization (RPO) is designed to discern between more and less preferred responses derived from both identical and related prompts. RPO has demonstrated a superior ability to align large language models with user preferences and to improve their adaptability during the training process.
  arXiv  Detail & Related papers  (2024-02-12T22:47:57Z)
• The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions [114.67699010359637]
  We analyze a large-scale collection of real user queries to GPT. We find that tasks such as design'' and planning'' are prevalent in user interactions but are largely neglected or different from traditional NLP benchmarks.
  arXiv  Detail & Related papers  (2023-10-19T02:12:17Z)
• Improving Knowledge Extraction from LLMs for Task Learning through Agent Analysis [4.055489363682198]
  Large language models (LLMs) offer significant promise as a knowledge source for task learning. Prompt engineering has been shown to be effective for eliciting knowledge from an LLM, but alone it is insufficient for acquiring relevant, situationally grounded knowledge for an embodied agent learning novel tasks. We describe a cognitive-agent approach, STARS, that extends and complements prompt engineering, mitigating its limitations and thus enabling an agent to acquire new task knowledge matched to its native language capabilities, embodiment, environment, and user preferences.
  arXiv  Detail & Related papers  (2023-06-11T20:50:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.