Open-Vocabulary Affordance Detection in 3D Point Clouds
- URL: http://arxiv.org/abs/2303.02401v5
- Date: Sun, 23 Jul 2023 08:31:15 GMT
- Title: Open-Vocabulary Affordance Detection in 3D Point Clouds
- Authors: Toan Nguyen, Minh Nhat Vu, An Vuong, Dzung Nguyen, Thieu Vo, Ngan Le,
Anh Nguyen
- Abstract summary: Open-Vocabulary Affordance Detection (OpenAD) method is capable of detecting an unbounded number of affordances in 3D point clouds.
Our proposed method enables zero-shot detection and can be able to detect previously unseen affordances.
- Score: 6.4274167612662465
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Affordance detection is a challenging problem with a wide variety of robotic
applications. Traditional affordance detection methods are limited to a
predefined set of affordance labels, hence potentially restricting the
adaptability of intelligent robots in complex and dynamic environments. In this
paper, we present the Open-Vocabulary Affordance Detection (OpenAD) method,
which is capable of detecting an unbounded number of affordances in 3D point
clouds. By simultaneously learning the affordance text and the point feature,
OpenAD successfully exploits the semantic relationships between affordances.
Therefore, our proposed method enables zero-shot detection and can be able to
detect previously unseen affordances without a single annotation example.
Intensive experimental results show that OpenAD works effectively on a wide
range of affordance detection setups and outperforms other baselines by a large
margin. Additionally, we demonstrate the practicality of the proposed OpenAD in
real-world robotic applications with a fast inference speed (~100ms). Our
project is available at https://openad2023.github.io.
Related papers
- Bridging the Gap Between End-to-End and Two-Step Text Spotting [88.14552991115207]
Bridging Text Spotting is a novel approach that resolves the error accumulation and suboptimal performance issues in two-step methods.
We demonstrate the effectiveness of the proposed method through extensive experiments.
arXiv Detail & Related papers (2024-04-06T13:14:04Z) - Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments [67.83787474506073]
We tackle the limitations of current LiDAR-based 3D object detection systems.
We introduce a universal textscFind n' Propagate approach for 3D OV tasks.
We achieve up to a 3.97-fold increase in Average Precision (AP) for novel object classes.
arXiv Detail & Related papers (2024-03-20T12:51:30Z) - Open-CRB: Towards Open World Active Learning for 3D Object Detection [40.80953254074535]
LiDAR-based 3D object detection has recently seen significant advancements through active learning (AL)
In real-world deployments where streaming point clouds may include unknown or novel objects, the ability of current AL methods to capture such objects remains unexplored.
This paper investigates a more practical and challenging research task: Open World Active Learning for 3D Object Detection (OWAL-3D)
arXiv Detail & Related papers (2023-10-16T13:32:53Z) - LS-VOS: Identifying Outliers in 3D Object Detections Using Latent Space
Virtual Outlier Synthesis [10.920640666237833]
LiDAR-based 3D object detectors have achieved unprecedented speed and accuracy in autonomous driving applications.
They are often biased toward high-confidence predictions or return detections where no real object is present.
We propose LS-VOS, a framework for identifying outliers in 3D object detections.
arXiv Detail & Related papers (2023-10-02T07:44:26Z) - KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection [48.66703222700795]
We resort to a novel kernel strategy to identify the most informative point clouds to acquire labels.
To accommodate both one-stage (i.e., SECOND) and two-stage detectors, we incorporate the classification entropy tangent and well trade-off between detection performance and the total number of bounding boxes selected for annotation.
Our results show that approximately 44% box-level annotation costs and 26% computational time are reduced compared to the state-of-the-art method.
arXiv Detail & Related papers (2023-07-16T04:27:03Z) - Exploring Active 3D Object Detection from a Generalization Perspective [58.597942380989245]
Uncertainty-based active learning policies fail to balance the trade-off between point cloud informativeness and box-level annotation costs.
We propose textscCrb, which hierarchically filters out the point clouds of redundant 3D bounding box labels.
Experiments show that the proposed approach outperforms existing active learning strategies.
arXiv Detail & Related papers (2023-01-23T02:43:03Z) - Prompt-driven efficient Open-set Semi-supervised Learning [52.30303262499391]
Open-set semi-supervised learning (OSSL) has attracted growing interest, which investigates a more practical scenario where out-of-distribution (OOD) samples are only contained in unlabeled data.
We propose a prompt-driven efficient OSSL framework, called OpenPrompt, which can propagate class information from labeled to unlabeled data with only a small number of trainable parameters.
arXiv Detail & Related papers (2022-09-28T16:25:08Z) - Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving [45.405303803618]
We investigate diversity-based active learning (AL) as a potential solution to alleviate the annotation burden.
We propose a novel acquisition function that enforces spatial and temporal diversity in the selected samples.
We demonstrate the effectiveness of the proposed method on the nuScenes dataset and show that it outperforms existing AL strategies significantly.
arXiv Detail & Related papers (2022-05-16T14:21:30Z) - Open-set Adversarial Defense [93.25058425356694]
We show that open-set recognition systems are vulnerable to adversarial attacks.
Motivated by this observation, we emphasize the need of an Open-Set Adrial Defense (OSAD) mechanism.
This paper proposes an Open-Set Defense Network (OSDN) as a solution to the OSAD problem.
arXiv Detail & Related papers (2020-09-02T04:35:33Z) - AFDet: Anchor Free One Stage 3D Object Detection [9.981769027320551]
High-efficiency point cloud 3D object detection is important for many robotics applications including autonomous driving.
Most previous works try to solve it using anchor-based detection methods which come with two drawbacks: post-processing is relatively complex and computationally expensive; tuning anchor parameters is tricky.
We are the first to address these drawbacks with an anchor free and Non-Maximum Suppression free one stage detector called AFDet.
arXiv Detail & Related papers (2020-06-23T00:15:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.