Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Cameras

• URL: http://arxiv.org/abs/2607.28293v1
• Date: Thu, 30 Jul 2026 14:37:30 GMT
• Title: Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Cameras
• Abstract summary: The paper proposes a reformulated version of hardware-aware neural architecture search.<n>It is endowed with a newly designed search space to integrate depth (D) information into small-sized deep networks.<n>It also includes a preprocessing layer to merge depth information with RGB data and make it compatible with conventional architectures.
• Score: 2.3096589301166843
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While depth sensors have the potential to complement RGB data for affordance segmentation in wearable robots, their usage seems to remain underexplored. The paper proposes two approaches: a reformulated version of hardware-aware neural architecture search, endowed with a newly designed search space to integrate depth (D) information into small-sized deep networks, and a dedicated fine-tuning approach, including a preprocessing layer to merge depth information with RGB data and make it compatible with conventional architectures. In both cases, those methods aim to generate solutions that benefit from modern (portable) hardware accelerators and overcome existing tiny-like approaches, which often fail to tackle critical scenarios due to the severe constraints set by the supporting hardware. Extensive experiments on a pair of real-world datasets demonstrate the effectiveness of the proposed method as compared with existing solutions. The approach presented in the paper generates, in most cases, solutions that identify the Pareto optimal front to balance generalization performance and hardware requirements. The paper also describes the supporting prototype, including a Jetson Nano board and a RealSense RGB-D camera. When considering the energy profile of the device, the overall system can attain real-time performances within an energy budget that is compatible with standard batteries, such as those used in smartphones.

Related papers

• HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection [75.406055413928]
  We propose a novel prompt-driven segment anything model (HyPSAM) for RGB-T SOD.<n> DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction.<n>Experiments on three public datasets demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-23T07:32:11Z)
• Power Battery Detection [91.99787495748218]
  Power batteries are essential components in electric vehicles, where internal structural defects can pose serious safety risks.<n>We conduct a comprehensive study on power battery detection (PBD), which aims to localize the dense endpoints of cathode and anode plates from X-ray images for quality inspection.<n>We present PBD5K, the first large-scale benchmark for this task, consisting of 5,000 X-ray images from nine battery types with fine-grained annotations and eight types of real-world visual interference.
  arXiv  Detail & Related papers  (2025-08-11T09:35:25Z)
• DiFuse-Net: RGB and Dual-Pixel Depth Estimation using Window Bi-directional Parallax Attention and Cross-modal Transfer Learning [1.456352735394398]
  DiFuse-Net is a novel decoupled network design for disentangled RGB and DP based depth estimation.<n>WBiPAM captures subtle DP disparity cues unique to smartphone cameras with small aperture.<n>A separate encoder extracts contextual information from the RGB image, and these features are fused to enhance depth prediction.
  arXiv  Detail & Related papers  (2025-06-17T16:49:27Z)
• DepthMatch: Semi-Supervised RGB-D Scene Parsing through Depth-Guided Regularization [43.974708665104565]
  We introduce DepthMatch, a semi-supervised learning framework that is specifically designed for RGB-D scene parsing.<n>We propose complementary patch mix-up augmentation to explore the latent relationships between texture and spatial features in RGB-D image pairs.<n>We also design a lightweight spatial prior injector to replace traditional complex fusion modules, improving the efficiency of heterogeneous feature fusion.
  arXiv  Detail & Related papers  (2025-05-26T14:26:31Z)
• Lightweight RGB-D Salient Object Detection from a Speed-Accuracy Tradeoff Perspective [54.91271106816616]
  Current RGB-D methods usually leverage large-scale backbones to improve accuracy but sacrifice efficiency.<n>We propose a Speed-Accuracy Tradeoff Network (SATNet) for Lightweight RGB-D SOD from three fundamental perspectives.<n> Concerning depth quality, we introduce the Depth Anything Model to generate high-quality depth maps.<n>For modality fusion, we propose a Decoupled Attention Module (DAM) to explore the consistency within and between modalities.<n>For feature representation, we develop a Dual Information Representation Module (DIRM) with a bi-directional inverted framework.
  arXiv  Detail & Related papers  (2025-05-07T19:37:20Z)
• Diffusion-based RGB-D Semantic Segmentation with Deformable Attention Transformer [11.648973329789973]
  We introduce a diffusion-based framework to address the RGB-D semantic segmentation problem.<n>We demonstrate that utilizing a Deformable Attention Transformer as the encoder to extract features from depth images effectively captures the characteristics of invalid regions in depth measurements.
  arXiv  Detail & Related papers  (2024-09-23T15:23:01Z)
• METER: a mobile vision transformer architecture for monocular depth estimation [0.0]
  We propose METER, a novel lightweight vision transformer architecture capable of achieving state of the art estimations. We provide a solution consisting of three alternative configurations of METER, a novel loss function to balance pixel estimation and reconstruction of image details, and a new data augmentation strategy to improve the overall final predictions.
  arXiv  Detail & Related papers  (2024-03-13T09:30:08Z)
• Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis [48.59382455101753]
  2D face recognition encounters challenges in unconstrained environments due to varying illumination, occlusion, and pose. Recent studies focus on RGB-D face recognition to improve robustness by incorporating depth information. In this work, we first construct a diverse depth dataset generated by 3D Morphable Models for depth model pre-training. Then, we propose a domain-independent pre-training framework that utilizes readily available pre-trained RGB and depth models to separately perform face recognition without needing additional paired data for retraining.
  arXiv  Detail & Related papers  (2024-03-11T09:12:24Z)
• Modality-Guided Subnetwork for Salient Object Detection [5.491692465987937]
  Most RGBD networks require multi-modalities from the input side and feed them separately through a two-stream design. We present in this paper a novel fusion design named modality-guided subnetwork (MGSnet) It has the following superior designs: 1) Our model works for both RGB and RGBD data, and dynamically estimating depth if not available.
  arXiv  Detail & Related papers  (2021-10-10T20:59:11Z)
• Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection [78.47767202232298]
  We propose a novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD. Two components are designed to implement the effective cross-modality interaction. Our network outperforms $15$ state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2021-08-04T11:24:42Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)
• Is Depth Really Necessary for Salient Object Detection? [50.10888549190576]
  We make the first attempt in realizing an unified depth-aware framework with only RGB information as input for inference. Not only surpasses the state-of-the-art performances on five public RGB SOD benchmarks, but also surpasses the RGBD-based methods on five benchmarks by a large margin.
  arXiv  Detail & Related papers  (2020-05-30T13:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.