Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN

• URL: http://arxiv.org/abs/2305.01910v1
• Date: Wed, 3 May 2023 05:57:29 GMT
• Title: Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN
• Authors: YuXuan Liu, Nikhil Mishra, Pieter Abbeel, Xi Chen
• Abstract summary: In this paper, we explore a class of distributional instance segmentation models using latent codes. For robotic picking applications, we propose a confidence mask method to achieve the high precision necessary. We show that our method can significantly reduce critical errors in robotic systems, including our newly released dataset of ambiguous scenes.
• Score: 77.0623472106488
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Object recognition and instance segmentation are fundamental skills in any robotic or autonomous system. Existing state-of-the-art methods are often unable to capture meaningful uncertainty in challenging or ambiguous scenes, and as such can cause critical errors in high-performance applications. In this paper, we explore a class of distributional instance segmentation models using latent codes that can model uncertainty over plausible hypotheses of object masks. For robotic picking applications, we propose a confidence mask method to achieve the high precision necessary in industrial use cases. We show that our method can significantly reduce critical errors in robotic systems, including our newly released dataset of ambiguous scenes in a robotic application. On a real-world apparel-picking robot, our method significantly reduces double pick errors while maintaining high performance.

Related papers

• Embodied Uncertainty-Aware Object Segmentation [38.52448300879023]
  We introduce uncertainty-aware object instance segmentation (UncOS) and demonstrate its usefulness for embodied interactive segmentation. We obtain a set of region-factored segmentation hypotheses together with confidence estimates by making multiple queries of large pre-trained models. The output can also serve as input to a belief-driven process for selecting robot actions to perturb the scene to reduce ambiguity.
  arXiv  Detail & Related papers  (2024-08-08T21:29:22Z)
• LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks [52.46420522934253]
  We introduce LoRA-Ensemble, a parameter-efficient deep ensemble method for self-attention networks. By employing a single pre-trained self-attention network with weights shared across all members, we train member-specific low-rank matrices for the attention projections. Our method exhibits superior calibration compared to explicit ensembles and achieves similar or better accuracy across various prediction tasks and datasets.
  arXiv  Detail & Related papers  (2024-05-23T11:10:32Z)
• A Causal Bayesian Network and Probabilistic Programming Based Reasoning Framework for Robot Manipulation Under Uncertainty [4.087774077861305]
  We propose a flexible and generalisable physics-informed causal Bayesian network (CBN) based framework for a robot. We demonstrate our framework's ability to: (1) predict manipulation outcomes with high accuracy (Pred Acc: 88.6%); and, (2) perform greedy next-best action selection with 94.2% task success rate.
  arXiv  Detail & Related papers  (2024-03-21T15:36:26Z)
• Towards Certified Probabilistic Robustness with High Accuracy [3.957941698534126]
  Adrial examples pose a security threat to many critical systems built on neural networks. How to build certifiably robust yet accurate neural network models remains an open problem. We propose a novel approach that aims to achieve both high accuracy and certified probabilistic robustness.
  arXiv  Detail & Related papers  (2023-09-02T09:39:47Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Formal Modelling for Multi-Robot Systems Under Uncertainty [11.21074891465253]
  We review modelling formalisms for multi-robot systems under uncertainty. We discuss how they can be used for planning, reinforcement learning, model checking, and simulation.
  arXiv  Detail & Related papers  (2023-05-26T15:23:35Z)
• $f$-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception [9.425514903472545]
  Existing approaches estimate uncertainty from neural network perception stacks by modifying network architectures, inference procedure, or loss functions. Our key insight is that calibration is only achieved by imposing constraints across multiple examples, such as those in a mini-batch. By enforcing the distribution of outputs of a neural network to resemble a target distribution by minimizing an $f$-divergence, we obtain significantly better-calibrated models compared to prior approaches.
  arXiv  Detail & Related papers  (2021-09-28T17:57:58Z)
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System [78.60415450507706]
  Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty.
  arXiv  Detail & Related papers  (2021-07-28T10:28:05Z)
• Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms [60.59764170868101]
  Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
  arXiv  Detail & Related papers  (2021-03-05T14:16:20Z)
• Where is my hand? Deep hand segmentation for visual self-recognition in humanoid robots [129.46920552019247]
  We propose the use of a Convolution Neural Network (CNN) to segment the robot hand from an image in an egocentric view. We fine-tuned the Mask-RCNN network for the specific task of segmenting the hand of the humanoid robot Vizzy.
  arXiv  Detail & Related papers  (2021-02-09T10:34:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.