TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

• URL: http://arxiv.org/abs/2606.14862v1
• Date: Fri, 12 Jun 2026 18:04:28 GMT
• Title: TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations
• Abstract summary: Natural language provides an intuitive way for humans to communicate preferences.<n>Recent progress in language-conditioned robot policies has shown that robots can successfully use language prompts to determine what task to perform.<n>We show that selecting robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences.
• Score: 0.9253666753150972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic systems that assist humans should be capable of adapting their behaviors to individual user preferences. For instance, users may want a robot arm to adjust the amount of force it applies while folding their laundry or cleaning furniture. Natural language provides an intuitive way for humans to communicate such preferences. Recent progress in language-conditioned robot policies has shown that robots can successfully use language prompts to determine what task to perform. However, extending the same approach to realize how the task should be performed requires detailed labels describing the preferences or styles of trajectories in the task data. Not only is collecting such annotations challenging, but conditioning directly on these labels may also fail to provide fine-grained control over a continuous range of behaviors. For example, it can be difficult to convey the exact force that a robot must apply through abstract instructions like "apply a bit more pressure than before". Therefore, in this work, we propose using language to reason over preferred behaviors instead of directly generating them. We first learn a structured latent representation that organizes user preferences according to differences in the corresponding trajectories. Then, given a preference prompt, we use a foundation model to interpret this latent space and choose a value that produces the desired behavior. Through both simulation and real-world experiments, we show that selecting robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences while requiring significantly fewer preference labels than language-conditioned policies.

Related papers

• Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences [53.353022588751585]
  We present Promptable Behaviors, a novel framework that facilitates efficient personalization of robotic agents to diverse human preferences. We introduce three distinct methods to infer human preferences by leveraging different types of interactions. We evaluate the proposed method in personalized object-goal navigation and flee navigation tasks in ProcTHOR and RoboTHOR.
  arXiv  Detail & Related papers  (2023-12-14T21:00:56Z)
• TidyBot: Personalized Robot Assistance with Large Language Models [46.629932362863386]
  A key challenge is determining the proper place to put each object. One person may prefer storing shirts in the drawer, while another may prefer them on the shelf. We show that robots can combine language-based planning and perception with the few-shot summarization capabilities of large language models.
  arXiv  Detail & Related papers  (2023-05-09T17:52:59Z)
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy [70.45420918526926]
  We present LILAC, a framework for incorporating and adapting to natural language corrections online during execution. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot. We show that our corrections-aware approach obtains higher task completion rates, and is subjectively preferred by users.
  arXiv  Detail & Related papers  (2023-01-06T15:03:27Z)
• Correcting Robot Plans with Natural Language Feedback [88.92824527743105]
  We explore natural language as an expressive and flexible tool for robot correction. We show that these transformations enable users to correct goals, update robot motions, and recover from planning errors. Our method makes it possible to compose multiple constraints and generalizes to unseen scenes, objects, and sentences in simulated environments and real-world environments.
  arXiv  Detail & Related papers  (2022-04-11T15:22:43Z)
• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances [119.29555551279155]
  Large language models can encode a wealth of semantic knowledge about the world. Such knowledge could be extremely useful to robots aiming to act upon high-level, temporally extended instructions expressed in natural language. We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions.
  arXiv  Detail & Related papers  (2022-04-04T17:57:11Z)
• Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers [33.7939079214046]
  We provide a flexible language-based interface for human-robot collaboration. We take advantage of recent advancements in the field of large language models to encode the user command. We train the model using imitation learning over a dataset containing robot trajectories modified by language commands.
  arXiv  Detail & Related papers  (2022-03-25T01:36:56Z)
• Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation [80.29069988090912]
  We study the problem of learning a range of vision-based manipulation tasks from a large offline dataset of robot interaction. We propose to leverage offline robot datasets with crowd-sourced natural language labels. We find that our approach outperforms both goal-image specifications and language conditioned imitation techniques by more than 25%.
  arXiv  Detail & Related papers  (2021-09-02T17:42:13Z)
• Translating Natural Language Instructions to Computer Programs for Robot Manipulation [0.6629765271909505]
  We propose translating the natural language instruction to a Python function which queries the scene by accessing the output of the object detector. We show that the proposed method performs better than training a neural network to directly predict the robot actions.
  arXiv  Detail & Related papers  (2020-12-26T07:57:55Z)
• Language Conditioned Imitation Learning over Unstructured Data [9.69886122332044]
  We present a method for incorporating free-form natural language conditioning into imitation learning. Our approach learns perception from pixels, natural language understanding, and multitask continuous control end-to-end as a single neural network. We show this dramatically improves language conditioned performance, while reducing the cost of language annotation to less than 1% of total data.
  arXiv  Detail & Related papers  (2020-05-15T17:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.