NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction

• URL: http://arxiv.org/abs/2109.13859v1
• Date: Wed, 22 Sep 2021 05:17:09 GMT
• Title: NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction
• Authors: Chahat Deep Singh, Nitin J. Sanket, Chethan M. Parameshwara, Cornelia Ferm\"uller, Yiannis Aloimonos
• Abstract summary: We present the first framework to segment unknown objects in a cluttered scene by repeatedly 'nudging' at the objects. We show an impressive average detection rate of over 86% on zero-shot objects.
• Score: 8.712677353734627
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in object segmentation have demonstrated that deep neural networks excel at object segmentation for specific classes in color and depth images. However, their performance is dictated by the number of classes and objects used for training, thereby hindering generalization to never seen objects or zero-shot samples. To exacerbate the problem further, object segmentation using image frames rely on recognition and pattern matching cues. Instead, we utilize the 'active' nature of a robot and their ability to 'interact' with the environment to induce additional geometric constraints for segmenting zero-shot samples. In this paper, we present the first framework to segment unknown objects in a cluttered scene by repeatedly 'nudging' at the objects and moving them to obtain additional motion cues at every step using only a monochrome monocular camera. We call our framework NudgeSeg. These motion cues are used to refine the segmentation masks. We successfully test our approach to segment novel objects in various cluttered scenes and provide an extensive study with image and motion segmentation methods. We show an impressive average detection rate of over 86% on zero-shot objects.

Related papers

• LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-training [13.985488693082981]
  We propose a self-supervised object discovery approach that leverages motion and appearance information to produce high-quality object segmentation masks. We demonstrate the effectiveness of our approach, named LOCATE, on multiple standard video object segmentation, image saliency detection, and object segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-08-22T07:27:09Z)
• Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping [52.03068246508119]
  We study learning object segmentation from unlabeled videos. We learn an image segmenter first in the loop of approximating optical flow with constant segment flow plus small within-segment residual flow. Our model surpasses the state-of-the-art by absolute gains of 7/9/5% on DAVIS16 / STv2 / FBMS59 respectively.
  arXiv  Detail & Related papers  (2023-04-17T07:18:21Z)
• Unsupervised Multi-object Segmentation by Predicting Probable Motion Patterns [92.80981308407098]
  We propose a new approach to learn to segment multiple image objects without manual supervision. The method can extract objects form still images, but uses videos for supervision. We show state-of-the-art unsupervised object segmentation performance on simulated and real-world benchmarks.
  arXiv  Detail & Related papers  (2022-10-21T17:57:05Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)
• The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos [59.12750806239545]
  We show that a video has different views of the same scene related by moving components, and the right region segmentation and region flow would allow mutual view synthesis. Our model starts with two separate pathways: an appearance pathway that outputs feature-based region segmentation for a single image, and a motion pathway that outputs motion features for a pair of images. By training the model to minimize view synthesis errors based on segment flow, our appearance and motion pathways learn region segmentation and flow estimation automatically without building them up from low-level edges or optical flows respectively.
  arXiv  Detail & Related papers  (2021-11-11T18:59:11Z)
• A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection [56.82077636126353]
  We take advantage of object-centric images to improve object detection in scene-centric images. We present a simple yet surprisingly effective framework to do so. Our approach can improve the object detection (and instance segmentation) accuracy of rare objects by 50% (and 33%) relatively.
  arXiv  Detail & Related papers  (2021-02-17T17:27:21Z)
• DyStaB: Unsupervised Object Segmentation via Dynamic-Static Bootstrapping [72.84991726271024]
  We describe an unsupervised method to detect and segment portions of images of live scenes that are seen moving as a coherent whole. Our method first partitions the motion field by minimizing the mutual information between segments. It uses the segments to learn object models that can be used for detection in a static image.
  arXiv  Detail & Related papers  (2020-08-16T22:05:13Z)
• Reinforced Coloring for End-to-End Instance Segmentation [10.73460247817528]
  We propose a novel iterative deep reinforcement learning agent that learns how to differentiate multiple objects in parallel. Our reward function for the trainable agent is designed to favor grouping pixels belonging to the same object using a graph coloring algorithm.
  arXiv  Detail & Related papers  (2020-05-14T15:15:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.