CARPE-ID: Continuously Adaptable Re-identification for Personalized Robot Assistance

• URL: http://arxiv.org/abs/2310.19413v2
• Date: Wed, 31 Jan 2024 14:31:23 GMT
• Title: CARPE-ID: Continuously Adaptable Re-identification for Personalized Robot Assistance
• Authors: Federico Rollo, Andrea Zunino, Nikolaos Tsagarakis, Enrico Mingo Hoffman, Arash Ajoudani
• Abstract summary: In today's Human-Robot Interaction (HRI) scenarios, a prevailing tendency exists to assume that the robot shall cooperate with the closest individual. We propose a person re-identification module based on continual visual adaptation techniques. We test the framework singularly using recorded videos in a laboratory environment and an HRI scenario by a mobile robot.
• Score: 16.948256303861022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In today's Human-Robot Interaction (HRI) scenarios, a prevailing tendency exists to assume that the robot shall cooperate with the closest individual or that the scene involves merely a singular human actor. However, in realistic scenarios, such as shop floor operations, such an assumption may not hold and personalized target recognition by the robot in crowded environments is required. To fulfil this requirement, in this work, we propose a person re-identification module based on continual visual adaptation techniques that ensure the robot's seamless cooperation with the appropriate individual even subject to varying visual appearances or partial or complete occlusions. We test the framework singularly using recorded videos in a laboratory environment and an HRI scenario, i.e., a person-following task by a mobile robot. The targets are asked to change their appearance during tracking and to disappear from the camera field of view to test the challenging cases of occlusion and outfit variations. We compare our framework with one of the state-of-the-art Multi-Object Tracking (MOT) methods and the results show that the CARPE-ID can accurately track each selected target throughout the experiments in all the cases (except two limit cases). At the same time, the s-o-t-a MOT has a mean of 4 tracking errors for each video.

Related papers

• OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation [35.97702591413093]
  We introduce OKAMI, a method that generates a manipulation plan from a single RGB-D video. OKAMI uses open-world vision models to identify task-relevant objects and retarget the body motions and hand poses separately.
  arXiv  Detail & Related papers  (2024-10-15T17:17:54Z)
• CtRNet-X: Camera-to-Robot Pose Estimation in Real-world Conditions Using a Single Camera [18.971816395021488]
  Markerless pose estimation methods have eliminated the need for time-consuming physical setups for camera-to-robot calibration. We propose a novel framework capable of estimating the robot pose with partially visible robot manipulators.
  arXiv  Detail & Related papers  (2024-09-16T16:22:43Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Exploring 3D Human Pose Estimation and Forecasting from the Robot's Perspective: The HARPER Dataset [52.22758311559]
  We introduce HARPER, a novel dataset for 3D body pose estimation and forecast in dyadic interactions between users and Spot. The key-novelty is the focus on the robot's perspective, i.e., on the data captured by the robot's sensors. The scenario underlying HARPER includes 15 actions, of which 10 involve physical contact between the robot and users.
  arXiv  Detail & Related papers  (2024-03-21T14:53:50Z)
• Revisit Human-Scene Interaction via Space Occupancy [55.67657438543008]
  Human-scene Interaction (HSI) generation is a challenging task and crucial for various downstream tasks. In this work, we argue that interaction with a scene is essentially interacting with the space occupancy of the scene from an abstract physical perspective. By treating pure motion sequences as records of humans interacting with invisible scene occupancy, we can aggregate motion-only data into a large-scale paired human-occupancy interaction database.
  arXiv  Detail & Related papers  (2023-12-05T12:03:00Z)
• Learning Video-Conditioned Policies for Unseen Manipulation Tasks [83.2240629060453]
  Video-conditioned Policy learning maps human demonstrations of previously unseen tasks to robot manipulation skills. We learn our policy to generate appropriate actions given current scene observations and a video of the target task. We validate our approach on a set of challenging multi-task robot manipulation environments and outperform state of the art.
  arXiv  Detail & Related papers  (2023-05-10T16:25:42Z)
• Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly [18.563562557565483]
  We propose the framework "Scene Editing as Teleoperation" (SEaT) Instead of controlling the robot, users focus on specifying the task's goal. A user can perform teleoperation without any expert knowledge of the robot hardware.
  arXiv  Detail & Related papers  (2021-10-09T04:22:21Z)
• Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos [59.58105314783289]
  Domain-agnostic Video Discriminator (DVD) learns multitask reward functions by training a discriminator to classify whether two videos are performing the same task. DVD can generalize by virtue of learning from a small amount of robot data with a broad dataset of human videos. DVD can be combined with visual model predictive control to solve robotic manipulation tasks on a real WidowX200 robot in an unseen environment from a single human demo.
  arXiv  Detail & Related papers  (2021-03-31T05:25:05Z)
• Few-Shot Visual Grounding for Natural Human-Robot Interaction [0.0]
  We propose a software architecture that segments a target object from a crowded scene, indicated verbally by a human user. At the core of our system, we employ a multi-modal deep neural network for visual grounding. We evaluate the performance of the proposed model on real RGB-D data collected from public scene datasets.
  arXiv  Detail & Related papers  (2021-03-17T15:24:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.