DA-Fusion: Deformable Attention-Based RGB-D Fusion Transformer for Unseen Object Instance Segmentation

• URL: http://arxiv.org/abs/2607.17754v1
• Date: Mon, 20 Jul 2026 09:47:16 GMT
• Title: DA-Fusion: Deformable Attention-Based RGB-D Fusion Transformer for Unseen Object Instance Segmentation
• Abstract summary: In logistics automation, precise segmentation of unseen objects is crucial for efficient robotic manipulation in cluttered environments.<n>Traditional RGB-based methods tend to over-segment objects due to their reliance on texture, while depth-based methods often under-segment by focusing primarily on geometric features.<n>We propose DA-Fusion, a deformable attention-based RGB-D fusion Transformer designed for unseen object instance segmentation.
• Score: 24.37584731810519
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In logistics automation, precise segmentation of unseen objects is crucial for efficient robotic manipulation in cluttered environments. Tasks such as bin-picking and shelf-picking require robust perception to handle occlusions, varying object shapes, and complex spatial arrangements. Traditional RGB-based methods tend to over-segment objects due to their reliance on texture, while depth-based methods often under-segment by focusing primarily on geometric features. To address these limitations, we propose DA-Fusion, a deformable attention-based RGB-D fusion Transformer designed for unseen object instance segmentation. DA-Fusion effectively combines the strengths of both RGB and depth data, enhancing segmentation accuracy in cluttered and multi-layered object environments. We also introduce the Object Clutter Bin Dataset (OCBD), a benchmark dataset specifically tailored for evaluating bin-picking scenarios in top-down views. Extensive evaluations demonstrate that DA-Fusion outperforms state-of-the-art methods across diverse environments, making it particularly suited for real-world logistics tasks.

Related papers

• Learning Spatial-Semantic Features for Robust Video Object Segmentation [108.045326229865]
  We propose a robust video object segmentation framework that learns spatial-semantic features and discriminative object queries.<n>The proposed method achieves state-of-the-art performance on benchmark data sets, including the DAVIS 2017 test (textbf87.8%), YoutubeVOS 2019 (textbf88.1%), MOSE val (textbf74.0%), and LVOS test (textbf73.0%)
  arXiv  Detail & Related papers  (2024-07-10T15:36:00Z)
• FusionVision: A comprehensive approach of 3D object reconstruction and segmentation from RGB-D cameras using YOLO and fast segment anything [1.5728609542259502]
  This paper introduces FusionVision, an exhaustive pipeline adapted for the robust 3D segmentation of objects in RGB-D imagery. The proposed FusionVision pipeline employs YOLO for identifying objects within the RGB image domain. The synergy between these components and their integration into 3D scene understanding ensures a cohesive fusion of object detection and segmentation.
  arXiv  Detail & Related papers  (2024-02-29T22:59:27Z)
• Segment Any Events via Weighted Adaptation of Pivotal Tokens [85.39087004253163]
  This paper focuses on the nuanced challenge of tailoring the Segment Anything Models (SAMs) for integration with event data. We introduce a multi-scale feature distillation methodology to optimize the alignment of token embeddings originating from event data with their RGB image counterparts.
  arXiv  Detail & Related papers  (2023-12-24T12:47:08Z)
• SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environments [67.34330257205525]
  In this work, we explore zero-shot instance segmentation (ZSIS) from RGB-D data to identify unseen objects in a semantic category-agnostic manner. We present a method that uses annotated objects to learn the objectness'' of pixels and generalize to unseen object categories in cluttered indoor environments.
  arXiv  Detail & Related papers  (2022-12-22T17:59:48Z)
• Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation [14.258456366985444]
  Recently, a popular solution is leveraging RGB-D features of large-scale synthetic data and applying the model to unseen real-world scenarios. We re-emphasize the adaptation process across Sim2Real domains in this paper. We propose a framework to conduct the Fully Test-time RGB-D Embeddings Adaptation (FTEA) based on parameters of the BatchNorm layer.
  arXiv  Detail & Related papers  (2022-04-21T02:35:20Z)
• Self-Supervised Representation Learning for RGB-D Salient Object Detection [93.17479956795862]
  We use Self-Supervised Representation Learning to design two pretext tasks: the cross-modal auto-encoder and the depth-contour estimation. Our pretext tasks require only a few and un RGB-D datasets to perform pre-training, which make the network capture rich semantic contexts. For the inherent problem of cross-modal fusion in RGB-D SOD, we propose a multi-path fusion module.
  arXiv  Detail & Related papers  (2021-01-29T09:16:06Z)
• Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation [67.88276573341734]
  We propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings. We further improve the segmentation accuracy with a new two-stage clustering algorithm.
  arXiv  Detail & Related papers  (2020-07-30T00:23:07Z)
• Unseen Object Instance Segmentation for Robotic Environments [67.88276573341734]
  We propose a method to segment unseen object instances in tabletop environments. UOIS-Net is comprised of two stages: first, it operates only on depth to produce object instance center votes in 2D or 3D. Surprisingly, our framework is able to learn from synthetic RGB-D data where the RGB is non-photorealistic.
  arXiv  Detail & Related papers  (2020-07-16T01:59:13Z)
• A Multi-Level Approach to Waste Object Segmentation [10.20384144853726]
  We address the problem of localizing waste objects from a color image and an optional depth image. Our method integrates the intensity and depth information at multiple levels of spatial granularity. We create a new RGBD waste object segmentation, MJU-Waste, that is made public to facilitate future research in this area.
  arXiv  Detail & Related papers  (2020-07-08T16:49:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.