DAP: Diffusion-based Affordance Prediction for Multi-modality Storage

• URL: http://arxiv.org/abs/2409.00499v1
• Date: Sat, 31 Aug 2024 16:37:29 GMT
• Title: DAP: Diffusion-based Affordance Prediction for Multi-modality Storage
• Abstract summary: We present a novel Diffusion-based Affordance Prediction pipeline for the multi-modal object storage problem. Our experiments demonstrate DAP's superior performance and training efficiency over the current state-of-the-art RPDiff. Our contribution fills a gap in robotic manipulation research by offering a solution that is both computationally efficient and capable of handling real-world variability.
• Score: 31.103726511999767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solving storage problem: where objects must be accurately placed into containers with precise orientations and positions, presents a distinct challenge that extends beyond traditional rearrangement tasks. These challenges are primarily due to the need for fine-grained 6D manipulation and the inherent multi-modality of solution spaces, where multiple viable goal configurations exist for the same storage container. We present a novel Diffusion-based Affordance Prediction (DAP) pipeline for the multi-modal object storage problem. DAP leverages a two-step approach, initially identifying a placeable region on the container and then precisely computing the relative pose between the object and that region. Existing methods either struggle with multi-modality issues or computation-intensive training. Our experiments demonstrate DAP's superior performance and training efficiency over the current state-of-the-art RPDiff, achieving remarkable results on the RPDiff benchmark. Additionally, our experiments showcase DAP's data efficiency in real-world applications, an advancement over existing simulation-driven approaches. Our contribution fills a gap in robotic manipulation research by offering a solution that is both computationally efficient and capable of handling real-world variability. Code and supplementary material can be found at: https://github.com/changhaonan/DPS.git.

Related papers

• DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models [55.01951088768769]
  DiffusionOPD is a new multi-task training paradigm for diffusion models based on Online Policy Distillation (OPD)<n>We show that DiffusionOPD consistently surpasses both multi-reward RL and cascade RL baselines in training efficiency and final performance.
  arXiv  Detail & Related papers  (2026-05-14T16:49:09Z)
• The Multi-AMR Buffer Storage, Retrieval, and Reshuffling Problem: Exact and Heuristic Approaches [0.5219568203653523]
  This paper introduces the Multi-AMR BSRRP, coordinating a robot fleet to manage concurrent reshuffling tasks.<n>We formulate a Binary Programming (IP) model to obtain exact solutions for benchmarking purposes.
  arXiv  Detail & Related papers  (2026-03-27T15:51:38Z)
• Beyond Weight Adaptation: Feature-Space Domain Injection for Cross-Modal Ship Re-Identification [3.6907522136316975]
  Cross-Modality Ship Re-Identification (CMS Re-ID) is critical for achieving all-day and all-weather maritime target tracking.<n>We explore the potential of Vision Foundation Models (VFMs) in bridging modality gaps.<n>We propose a novel PEFT strategy termed Domain Representation Injection (DRI)
  arXiv  Detail & Related papers  (2025-12-24T02:30:23Z)
• PAGen: Phase-guided Amplitude Generation for Domain-adaptive Object Detection [15.55359477953804]
  Unsupervised domain adaptation (UDA) greatly facilitates the deployment of neural networks across diverse environments.<n>We present a simple yet effective UDA method that learns to adapt image styles in the frequency domain to reduce the discrepancy between source and target domains.
  arXiv  Detail & Related papers  (2025-11-27T02:22:37Z)
• Boomda: Balanced Multi-objective Optimization for Multimodal Domain Adaptation [17.15772103162049]
  A popular solution is unsupervised domain adaptation, which has been extensively studied in unimodal settings.<n>In this paper, we investigate heterogeneous multimodal domain adaptation, where the primary challenge is the varying domain shifts of different modalities.<n>By exploiting the properties specific to our model, the problem can be simplified to a quadratic programming problem.<n>The proposed method features textbfBalanced multi-text-bfobjective textbfoptimization for textbfmultimodal textbfdomain textbf
  arXiv  Detail & Related papers  (2025-11-11T12:03:45Z)
• DoMIX: An Efficient Framework for Exploiting Domain Knowledge in Fine-Tuning [15.985923682239358]
  Domain-Adaptive Pre-training (DAP) has recently gained attention for its effectiveness in fine-tuning pre-trained models.<n>Existing continual DAP methods face several limitations.<n>We propose Domix, a novel approach that addresses these challenges by leveraging LoRA modules.
  arXiv  Detail & Related papers  (2025-07-03T04:13:01Z)
• Feature Fusion and Knowledge-Distilled Multi-Modal Multi-Target Detection [2.295863158976069]
  We propose a feature fusion and knowledge-distilled framework for multi-modal MTD.<n>We formulate the problem as a posterior probability optimization task, which is solved through a multi-stage training pipeline.<n> Experimental results demonstrate that our student model achieves approximately 95% of the teacher model's mean Average Precision.
  arXiv  Detail & Related papers  (2025-05-31T03:11:44Z)
• Domain Adaptation and Entanglement: an Optimal Transport Perspective [86.24617989187988]
  Current machine learning systems are brittle in the face of distribution shifts (DS), where the target distribution that the system is tested on differs from the source distribution used to train the system. For deep neural networks, a popular framework for unsupervised domain adaptation (UDA) is domain matching, in which algorithms try to align the marginal distributions in the feature or output space. In this paper, we derive new bounds based on optimal transport that analyze the UDA problem.
  arXiv  Detail & Related papers  (2025-03-11T08:10:03Z)
• From Holistic to Localized: Local Enhanced Adapters for Efficient Visual Instruction Fine-Tuning [102.18178065928426]
  Efficient Visual Instruction Fine-Tuning (EVIT) seeks to adapt Multimodal Large Language Models (MLLMs) to downstream tasks with minimal computational overhead.<n>We propose the Dual Low-Rank Adaptation (Dual-LoRA), a holistic-to-local framework that enhances the adapter's capacity to address data conflict.
  arXiv  Detail & Related papers  (2024-11-19T11:03:09Z)
• POMONAG: Pareto-Optimal Many-Objective Neural Architecture Generator [4.09225917049674]
  Transferable NAS has emerged, generalizing the search process from dataset-dependent to task-dependent. This paper introduces POMONAG, extending DiffusionNAG via a many-optimal diffusion process. Results were validated on two search spaces -- NAS201 and MobileNetV3 -- and evaluated across 15 image classification datasets.
  arXiv  Detail & Related papers  (2024-09-30T16:05:29Z)
• SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalities [55.87169702896249]
  Unsupervised Domain Adaptation (DA) consists of adapting a model trained on a labeled source domain to perform well on an unlabeled target domain with some data distribution shift. We present a complete and fair evaluation of existing shallow algorithms, including reweighting, mapping, and subspace alignment. Our benchmark highlights the importance of realistic validation and provides practical guidance for real-life applications.
  arXiv  Detail & Related papers  (2024-07-16T12:52:29Z)
• DANCE: Dual-View Distribution Alignment for Dataset Condensation [39.08022095906364]
  We propose a new DM-based method named Dual-view distribution AligNment for dataset CondEnsation (DANCE) Specifically, from the inner-class view, we construct multiple "middle encoders" to perform pseudo long-term distribution alignment. While from the inter-class view, we use the expert models to perform distribution calibration.
  arXiv  Detail & Related papers  (2024-06-03T07:22:17Z)
• AMFD: Distillation via Adaptive Multimodal Fusion for Multispectral Pedestrian Detection [23.91870504363899]
  Double-stream networks in multispectral detection employ two separate feature extraction branches for multi-modal data. This has hindered the widespread employment of multispectral pedestrian detection in embedded devices for autonomous systems. We introduce the Adaptive Modal Fusion Distillation (AMFD) framework, which can fully utilize the original modal features of the teacher network.
  arXiv  Detail & Related papers  (2024-05-21T17:17:17Z)
• Multimodal 3D Object Detection on Unseen Domains [37.142470149311904]
  Domain adaptation approaches assume access to unannotated samples from the test distribution to address this problem. We propose CLIX$text3D$, a multimodal fusion and supervised contrastive learning framework for 3D object detection. We show that CLIX$text3D$ yields state-of-the-art domain generalization performance under multiple dataset shifts.
  arXiv  Detail & Related papers  (2024-04-17T21:47:45Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• PM-DETR: Domain Adaptive Prompt Memory for Object Detection with Transformers [25.812325027602252]
  Transformer-based detectors (i.e., DETR) have demonstrated impressive performance on end-to-end object detection. transferring DETR to different data distributions may lead to a significant performance degradation. We propose a hierarchical Prompt Domain Memory (PDM) for adapting detection transformers to different distributions.
  arXiv  Detail & Related papers  (2023-07-01T12:02:24Z)
• Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation [86.02485817444216]
  We introduce Multi-Prompt Alignment (MPA), a simple yet efficient framework for multi-source UDA. MPA denoises the learned prompts through an auto-encoding process and aligns them by maximizing the agreement of all the reconstructed prompts. Experiments show that MPA achieves state-of-the-art results on three popular datasets with an impressive average accuracy of 54.1% on DomainNet.
  arXiv  Detail & Related papers  (2022-09-30T03:40:10Z)
• Learning Feature Decomposition for Domain Adaptive Monocular Depth Estimation [51.15061013818216]
  Supervised approaches have led to great success with the advance of deep learning, but they rely on large quantities of ground-truth depth annotations. Unsupervised domain adaptation (UDA) transfers knowledge from labeled source data to unlabeled target data, so as to relax the constraint of supervised learning. We propose a novel UDA method for MDE, referred to as Learning Feature Decomposition for Adaptation (LFDA), which learns to decompose the feature space into content and style components.
  arXiv  Detail & Related papers  (2022-07-30T08:05:35Z)
• RoMA: Robust Model Adaptation for Offline Model-based Optimization [115.02677045518692]
  We consider the problem of searching an input maximizing a black-box objective function given a static dataset of input-output queries. A popular approach to solving this problem is maintaining a proxy model that approximates the true objective function. Here, the main challenge is how to avoid adversarially optimized inputs during the search.
  arXiv  Detail & Related papers  (2021-10-27T05:37:12Z)
• Regressive Domain Adaptation for Unsupervised Keypoint Detection [67.2950306888855]
  Domain adaptation (DA) aims at transferring knowledge from a labeled source domain to an unlabeled target domain. We present a method of regressive domain adaptation (RegDA) for unsupervised keypoint detection. Our method brings large improvement by 8% to 11% in terms of PCK on different datasets.
  arXiv  Detail & Related papers  (2021-03-10T16:45:22Z)
• Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search [65.51181219410763]
  One-step methods have been developed to handle pedestrian detection and identification sub-tasks using a single network. There are two major challenges in the current one-step approaches. We propose a decoupled and memory-reinforced network (DMRNet) to overcome these problems.
  arXiv  Detail & Related papers  (2021-02-22T06:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.