Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intents

• URL: http://arxiv.org/abs/2406.11047v1
• Date: Sun, 16 Jun 2024 19:13:01 GMT
• Title: Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intents
• Authors: Chandran Nandkumar, Luka Peternel,
• Abstract summary: This paper presents the design and evaluation of a novel multi-level LLM interface for supermarket robots. We compare this approach to a specialized GPT model powered by GPT-4 Turbo. We find statistically significant improvements in four key areas: performance, user satisfaction, user-agent partnership, and self-image enhancement.
• Score: 46.623273455512106
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This paper presents the design and evaluation of a novel multi-level LLM interface for supermarket robots to assist customers. The proposed interface allows customers to convey their needs through both generic and specific queries. While state-of-the-art systems like OpenAI's GPTs are highly adaptable and easy to build and deploy, they still face challenges such as increased response times and limitations in strategic control of the underlying model for tailored use-case and cost optimization. Driven by the goal of developing faster and more efficient conversational agents, this paper advocates for using multiple smaller, specialized LLMs fine-tuned to handle different user queries based on their specificity and user intent. We compare this approach to a specialized GPT model powered by GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) and qualitative participant feedback in a counterbalanced within-subjects experiment. Our findings show that our multi-LLM chatbot architecture outperformed the benchmarked GPT model across all 13 measured criteria, with statistically significant improvements in four key areas: performance, user satisfaction, user-agent partnership, and self-image enhancement. The paper also presents a method for supermarket robot navigation by mapping the final chatbot response to correct shelf numbers, enabling the robot to sequentially navigate towards the respective products, after which lower-level robot perception, control, and planning can be used for automated object retrieval. We hope this work encourages more efforts into using multiple, specialized smaller models instead of relying on a single powerful, but more expensive and slower model.

Related papers

• SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation [82.61572106180705]
  This paper presents a unified approach using vision-language models (VLMs) to improve keypoint prediction across various garment categories. We created a large-scale synthetic dataset using advanced simulation techniques, allowing scalable training without extensive real-world data. Experimental results indicate that the VLM-based method significantly enhances keypoint detection accuracy and task success rates.
  arXiv  Detail & Related papers  (2024-09-26T17:26:16Z)
• D-RMGPT: Robot-assisted collaborative tasks driven by large multimodal models [0.0]
  Detection-Robot Management GPT (D-RMGPT) is a robot-assisted assembly planner based on Large Multimodal Models (LMM) It can assist inexperienced operators in assembly tasks without requiring any markers or previous training. It achieves an assembly success rate of 83% while reducing the assembly time for inexperienced operators by 33% compared to the manual process.
  arXiv  Detail & Related papers  (2024-08-21T16:34:21Z)
• Rethinking Annotator Simulation: Realistic Evaluation of Whole-Body PET Lesion Interactive Segmentation Methods [26.55942230051388]
  We evaluate interactive segmentation models through either real user studies or simulated annotators. Real user studies are expensive and often limited in scale, while simulated annotators, also known as robot users, tend to overestimate model performance. We propose a more realistic robot user that reduces the user shift by incorporating human factors such as click variation and inter-annotator disagreement.
  arXiv  Detail & Related papers  (2024-04-02T10:19:17Z)
• ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation [30.693616802332745]
  This paper presents a novel benchmark, AssistGUI, to evaluate whether models are capable of manipulating the mouse and keyboard on the Windows platform in response to user-requested tasks. We propose an advanced Actor-Critic framework, which incorporates a sophisticated GUI driven by an AI agent and adept at handling lengthy procedural tasks.
  arXiv  Detail & Related papers  (2023-12-20T15:28:38Z)
• Unlocking the Potential of User Feedback: Leveraging Large Language Model as User Simulator to Enhance Dialogue System [65.93577256431125]
  We propose an alternative approach called User-Guided Response Optimization (UGRO) to combine it with a smaller task-oriented dialogue model. This approach uses LLM as annotation-free user simulator to assess dialogue responses, combining them with smaller fine-tuned end-to-end TOD models. Our approach outperforms previous state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2023-06-16T13:04:56Z)
• AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation [50.737355245505334]
  We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks. The resulting dataset AlphaBlock consists of 35 comprehensive high-level tasks of multi-step text plans and paired observation.
  arXiv  Detail & Related papers  (2023-05-30T09:54:20Z)
• AutoML-GPT: Automatic Machine Learning with GPT [74.30699827690596]
  We propose developing task-oriented prompts and automatically utilizing large language models (LLMs) to automate the training pipeline. We present the AutoML-GPT, which employs GPT as the bridge to diverse AI models and dynamically trains models with optimized hyper parameters. This approach achieves remarkable results in computer vision, natural language processing, and other challenging areas.
  arXiv  Detail & Related papers  (2023-05-04T02:09:43Z)
• Model Predictive Control for Fluid Human-to-Robot Handovers [50.72520769938633]
  Planning motions that take human comfort into account is not a part of the human-robot handover process. We propose to generate smooth motions via an efficient model-predictive control framework. We conduct human-to-robot handover experiments on a diverse set of objects with several users.
  arXiv  Detail & Related papers  (2022-03-31T23:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.