Open-vocabulary 3D scene perception in industrial environments

• URL: http://arxiv.org/abs/2602.19823v1
• Date: Mon, 23 Feb 2026 13:22:51 GMT
• Title: Open-vocabulary 3D scene perception in industrial environments
• Abstract summary: Recent open-vocabulary methods, leveraging 2D Vision-Language Foundation Models (VLFMs), target this task.<n>We first demonstrate that such models fail to generalize, performing poorly on common industrial objects.<n>We propose a training-free, open-vocabulary 3D perception pipeline that overcomes this limitation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autonomous vision applications in production, intralogistics, or manufacturing environments require perception capabilities beyond a small, fixed set of classes. Recent open-vocabulary methods, leveraging 2D Vision-Language Foundation Models (VLFMs), target this task but often rely on class-agnostic segmentation models pre-trained on non-industrial datasets (e.g., household scenes). In this work, we first demonstrate that such models fail to generalize, performing poorly on common industrial objects. Therefore, we propose a training-free, open-vocabulary 3D perception pipeline that overcomes this limitation. Instead of using a pre-trained model to generate instance proposals, our method simply generates masks by merging pre-computed superpoints based on their semantic features. Following, we evaluate the domain-adapted VLFM "IndustrialCLIP" on a representative 3D industrial workshop scene for open-vocabulary querying. Our qualitative results demonstrate successful segmentation of industrial objects.

Related papers

• Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines [50.471211036005286]
  We introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M)<n> MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection.<n>We propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations.
  arXiv  Detail & Related papers  (2026-06-06T03:06:10Z)
• FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios [58.34124792457706]
  The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution.<n>Progress is hindered by data scarcity and a lack of fine-grained domain semantics in existing datasets.<n>We first construct a high-quality dataset that combines realworld 2D images and 3D point clouds, annotated with fine-grained domain semantics.<n>We then evaluate 18 state-of-the-art MLLMs across three manufacturing tasks, namely verification, structural surface inspection, and assembly verification, revealing significant performance gaps.
  arXiv  Detail & Related papers  (2026-04-08T12:23:27Z)
• From Observation to Action: Latent Action-based Primitive Segmentation for VLA Pre-training in Industrial Settings [53.09342573704396]
  We present a novel framework to unlock vast unlabeled human demonstration data from continuous industrial video streams for Vision-Language-Action (VLA) model pre-training.<n>Our method first trains a lightweight motion tokenizer to encode motion dynamics, then employs an unsupervised action segmenter to discover and segment semantically coherent action primitives.<n>This is the first fully automated end-to-end system for extracting and organizing VLA pre-training data from unstructured industrial videos, offering a scalable solution for embodied AI integration in manufacturing.
  arXiv  Detail & Related papers  (2025-11-26T14:19:44Z)
• IADGPT: Unified LVLM for Few-Shot Industrial Anomaly Detection, Localization, and Reasoning via In-Context Learning [18.078896149087576]
  Few-Shot Industrial Anomaly Detection (FS-IAD) has important applications in automating industrial quality inspection.<n>We propose a unified framework, IADGPT, designed to perform FS-IAD in a human-like manner.<n>We present a new dataset comprising 100K images across 400 diverse industrial product categories with extensive-level textual annotations.
  arXiv  Detail & Related papers  (2025-08-14T14:24:47Z)
• Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding [11.222744122842023]
  We introduce a plug-and-play module that implicitly incorporates 3D geometry features into Vision-Language-Action (VLA) models.<n>Our method significantly improves the performance of state-of-the-art VLA models across diverse scenarios.
  arXiv  Detail & Related papers  (2025-07-01T04:05:47Z)
• 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds [81.14476072159049]
  3D Affordance detection is a challenging problem with broad applications on various robotic tasks.<n>We reformulate the traditional affordance detection paradigm into textit Reasoning Affordance (IRAS) task.<n>We propose 3D-ADLLM, a framework designed for reasoning affordance detection in 3D open-scene.
  arXiv  Detail & Related papers  (2025-02-27T12:29:44Z)
• Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation [68.81887041766373]
  We introduce a diffusion-based paradigm for domain-generalized 9-DoF object pose estimation.<n>We propose an effective diffusion model to redefine 9-DoF object pose estimation from a generative perspective.<n>We show that our method achieves state-of-the-art domain generalization performance.
  arXiv  Detail & Related papers  (2025-02-04T17:46:34Z)
• Industrial Language-Image Dataset (ILID): Adapting Vision Foundation Models for Industrial Settings [0.0]
  We introduce a pipeline to generate the Industrial Language-Image dataset (ILID) based on web-crawled data. We demonstrate effective self-supervised transfer learning and discussing downstream tasks after training on the cheaply acquired ILID.
  arXiv  Detail & Related papers  (2024-06-14T00:06:52Z)
• Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understanding [41.96929575241655]
  We introduce OV-SAM3D, a training-free method for understanding open-vocabulary 3D scenes. This framework is designed to perform understanding tasks for any 3D scene without requiring prior knowledge of the scene. Empirical evaluations on the ScanNet200 and nuScenes datasets demonstrate that our approach surpasses existing open-vocabulary methods in unknown open-world environments.
  arXiv  Detail & Related papers  (2024-05-24T14:07:57Z)
• Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment [55.11291053011696]
  This work presents a framework for dealing with 3D scene understanding when the labeled scenes are quite limited.<n>To extract knowledge for novel categories from the pre-trained vision-language models, we propose a hierarchical feature-aligned pre-training and knowledge distillation strategy.<n>In the limited reconstruction case, our proposed approach, termed WS3D++, ranks 1st on the large-scale ScanNet benchmark.
  arXiv  Detail & Related papers  (2023-12-01T15:47:04Z)
• Diffusion Models for Open-Vocabulary Segmentation [79.02153797465324]
  OVDiff is a novel method that leverages generative text-to-image diffusion models for unsupervised open-vocabulary segmentation. It relies solely on pre-trained components and outputs the synthesised segmenter directly, without training.
  arXiv  Detail & Related papers  (2023-06-15T17:51:28Z)
• Weakly Supervised 3D Open-vocabulary Segmentation [104.07740741126119]
  We tackle the challenges in 3D open-vocabulary segmentation by exploiting pre-trained foundation models CLIP and DINO in a weakly supervised manner. We distill the open-vocabulary multimodal knowledge and object reasoning capability of CLIP and DINO into a neural radiance field (NeRF) A notable aspect of our approach is that it does not require any manual segmentation annotations for either the foundation models or the distillation process.
  arXiv  Detail & Related papers  (2023-05-23T14:16:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.