Localized Interactive Instance Segmentation

• URL: http://arxiv.org/abs/2010.09140v2
• Date: Tue, 20 Oct 2020 09:57:07 GMT
• Title: Localized Interactive Instance Segmentation
• Authors: Soumajit Majumder, Angela Yao
• Abstract summary: We propose a clicking scheme wherein user interactions are restricted to the proximity of the object. We demonstrate the effectiveness of our proposed clicking scheme and localization strategy through detailed experimentation.
• Score: 24.55415554455844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In current interactive instance segmentation works, the user is granted a free hand when providing clicks to segment an object; clicks are allowed on background pixels and other object instances far from the target object. This form of interaction is highly inconsistent with the end goal of efficiently isolating objects of interest. In our work, we propose a clicking scheme wherein user interactions are restricted to the proximity of the object. In addition, we propose a novel transformation of the user-provided clicks to generate a weak localization prior on the object which is consistent with image structures such as edges, textures etc. We demonstrate the effectiveness of our proposed clicking scheme and localization strategy through detailed experimentation in which we raise state-of-the-art on several standard interactive segmentation benchmarks.

Related papers

• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [59.87033229815062]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered. Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics. We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)
• Learning from Exemplars for Interactive Image Segmentation [15.37506525730218]
  We introduce novel interactive segmentation frameworks for both a single object and multiple objects in the same category. Our model reduces users' labor by around 15%, requiring two fewer clicks to achieve target IoUs 85% and 90%.
  arXiv  Detail & Related papers  (2024-06-17T12:38:01Z)
• AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation [32.63772366307106]
  We introduce AGILE3D, an efficient, attention-based model that supports simultaneous segmentation of multiple 3D objects. Our core idea is to encode user clicks as spatial-temporal queries and enable explicit interactions between click queries and the 3D scene. In experiments with four different 3D point cloud datasets, AGILE3D sets a new state-of-the-art.
  arXiv  Detail & Related papers  (2023-06-01T17:59:10Z)
• SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network [70.89436857471887]
  We propose an interactive distractor selection method that is optimized to achieve the task with just a single click. Our method surpasses the precision and recall achieved by the traditional method of running panoptic segmentation. Our experiments demonstrate that the model can effectively and accurately segment unknown distracting objects interactively and in groups.
  arXiv  Detail & Related papers  (2023-05-28T04:05:24Z)
• Learning-based Relational Object Matching Across Views [63.63338392484501]
  We propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network.
  arXiv  Detail & Related papers  (2023-05-03T19:36:51Z)
• DynaMITe: Dynamic Query Bootstrapping for Multi-object Interactive Segmentation Transformer [58.95404214273222]
  Most state-of-the-art instance segmentation methods rely on large amounts of pixel-precise ground-truth for training. We introduce a more efficient approach, called DynaMITe, in which we represent user interactions as-temporal queries. Our architecture also alleviates any need to re-compute image features during refinement, and requires fewer interactions for segmenting multiple instances in a single image.
  arXiv  Detail & Related papers  (2023-04-13T16:57:02Z)
• Contour-based Interactive Segmentation [4.164728134421114]
  We consider a natural form of user interaction as a loose contour, and introduce a contour-based interactive segmentation method. We demonstrate that a single contour provides the same accuracy as multiple clicks, thus reducing the required amount of user interactions.
  arXiv  Detail & Related papers  (2023-02-13T13:35:26Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)
• FAIRS -- Soft Focus Generator and Attention for Robust Object Segmentation from Extreme Points [70.65563691392987]
  We present a new approach to generate object segmentation from user inputs in the form of extreme points and corrective clicks. We demonstrate our method's ability to generate high-quality training data as well as its scalability in incorporating extreme points, guiding clicks, and corrective clicks in a principled manner.
  arXiv  Detail & Related papers  (2020-04-04T22:25:47Z)
• Instance Segmentation of Visible and Occluded Regions for Finding and Picking Target from a Pile of Objects [25.836334764387498]
  We present a robotic system for picking a target from a pile of objects that is capable of finding and grasping the target object. We extend an existing instance segmentation model with a novel relook' architecture, in which the model explicitly learns the inter-instance relationship. Also, by using image synthesis, we make the system capable of handling new objects without human annotations.
  arXiv  Detail & Related papers  (2020-01-21T12:28:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.