EfficientPPS: Part-aware Panoptic Segmentation of Transparent Objects for Robotic Manipulation

• URL: http://arxiv.org/abs/2312.13906v1
• Date: Thu, 21 Dec 2023 14:51:23 GMT
• Title: EfficientPPS: Part-aware Panoptic Segmentation of Transparent Objects for Robotic Manipulation
• Authors: Benjamin Alt, Minh Dang Nguyen, Andreas Hermann, Darko Katic, Rainer J\"akel, R\"udiger Dillmann, Eric Sax
• Abstract summary: We present EfficientPPS, a neural architecture for part-aware panoptic segmentation. It provides robots with semantically rich visual information for grasping and ma-nipulation tasks. We also present an unsupervised data collection and labelling method to reduce the need for human involvement.
• Score: 1.0819401241801996
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The use of autonomous robots for assistance tasks in hospitals has the potential to free up qualified staff and im-prove patient care. However, the ubiquity of deformable and transparent objects in hospital settings poses signif-icant challenges to vision-based perception systems. We present EfficientPPS, a neural architecture for part-aware panoptic segmentation that provides robots with semantically rich visual information for grasping and ma-nipulation tasks. We also present an unsupervised data collection and labelling method to reduce the need for human involvement in the training process. EfficientPPS is evaluated on a dataset containing real-world hospital objects and demonstrated to be robust and efficient in grasping transparent transfusion bags with a collaborative robot arm.

Related papers

• Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models [53.22792173053473]
  We introduce an interactive robotic manipulation framework called Polaris. Polaris integrates perception and interaction by utilizing GPT-4 alongside grounded vision models. We propose a novel Synthetic-to-Real (Syn2Real) pose estimation pipeline.
  arXiv  Detail & Related papers  (2024-08-15T06:40:38Z)
• Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition [48.65867987106428]
  We introduce a novel system for joint learning between human operators and robots. It enables human operators to share control of a robot end-effector with a learned assistive agent. It reduces the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks.
  arXiv  Detail & Related papers  (2024-06-29T03:37:29Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• From Detection to Action Recognition: An Edge-Based Pipeline for Robot Human Perception [5.262840821732319]
  Service robots rely on Human Action Recognition (HAR) to interpret human actions and intentions. We propose an end-to-end pipeline that encompasses the entire process, starting from human detection and tracking, leading to action recognition.
  arXiv  Detail & Related papers  (2023-12-06T13:10:02Z)
• Self-trained Panoptic Segmentation [0.0]
  Panoptic segmentation is an important computer vision task which combines semantic and instance segmentation. Recent advancements in self-supervised learning approaches have shown great potential in leveraging synthetic and unlabelled data to generate pseudo-labels. The aim of this work is to develop a framework to perform embedding-based self-supervised panoptic segmentation using self-training in a synthetic-to-real domain adaptation problem setting.
  arXiv  Detail & Related papers  (2023-11-17T17:06:59Z)
• Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing [88.99939660183881]
  We propose a framework for autonomous robotic navigation for subretinal injection. Our method consists of an instrument pose estimation method, an online registration between the robotic and the i OCT system, and trajectory planning tailored for navigation to an injection target. Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method.
  arXiv  Detail & Related papers  (2023-01-17T21:41:21Z)
• Learning Visually Guided Latent Actions for Assistive Teleoperation [9.75385535829762]
  We develop assistive robots that condition their latent embeddings on visual inputs. We show that incorporating object detectors pretrained on small amounts of cheap, easy-to-collect structured data enables i) accurately recognizing the current context and ii) generalizing control embeddings to new objects and tasks.
  arXiv  Detail & Related papers  (2021-05-02T23:58:28Z)
• Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition [54.749127627191655]
  The ability to recognize human partners is an important social skill to build personalized and long-term human-robot interactions. Deep learning networks have achieved state-of-the-art results and demonstrated to be suitable tools to address such a task. One solution is to make robots learn from their first-hand sensory data with self-supervision.
  arXiv  Detail & Related papers  (2021-03-16T13:50:24Z)
• Careful with That! Observation of Human Movements to Estimate Objects Properties [106.925705883949]
  We focus on the features of human motor actions that communicate insights on the weight of an object. Our final goal is to enable a robot to autonomously infer the degree of care required in object handling.
  arXiv  Detail & Related papers  (2021-03-02T08:14:56Z)
• Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery [10.562627972607892]
  We show that it may be possible to use robot kinematic data coupled with laparoscopic images to alleviate the labelling problem. We propose a new deep learning based model for parallel processing of both laparoscopic and simulation images.
  arXiv  Detail & Related papers  (2020-07-17T16:33:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.