P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models

• URL: http://arxiv.org/abs/2604.15703v1
• Date: Fri, 17 Apr 2026 05:18:22 GMT
• Title: P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models
• Abstract summary: Prototypical Point-level Prompt Tuning (P$3$T) is a parameter-efficient prompt tuning method for 3D vision-language models (VLMs)<n>P$3$T consists of two components: 1) textitPoint Prompter, which generates instance-aware point-level prompts for the input point cloud, and 2) textitText Prompter, which employs learnable prompts into the input text instead of hand-crafted ones.<n>Our method matches or outperforms full fine-tuning in classification and few-shot learning, and further exhibits robust generalization under data
• Score: 15.665642095045305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rise of pre-trained models in the 3D point cloud domain for a wide range of real-world applications, adapting them to downstream tasks has become increasingly important. However, conventional full fine-tuning methods are computationally expensive and storage-intensive. Although prompt tuning has emerged as an efficient alternative, it often suffers from overfitting, thereby compromising generalization capability. To address this issue, we propose Prototypical Point-level Prompt Tuning (P$^3$T), a parameter-efficient prompt tuning method designed for pre-trained 3D vision-language models (VLMs). P$^3$T consists of two components: 1) \textit{Point Prompter}, which generates instance-aware point-level prompts for the input point cloud, and 2) \textit{Text Prompter}, which employs learnable prompts into the input text instead of hand-crafted ones. Since both prompters operate directly on input data, P$^3$T enables task-specific adaptation of 3D VLMs without sacrificing generalizability. Furthermore, to enhance embedding space alignment, which is key to fine-tuning 3D VLMs, we introduce a prototypical loss that reduces intra-category variance. Extensive experiments demonstrate that our method matches or outperforms full fine-tuning in classification and few-shot learning, and further exhibits robust generalization under data shift in the cross-dataset setting. The code is available at \textcolor{violet}{https://github.com/gyjung975/P3T}.

Related papers

• PCT-Prompt: A Prompt-Guided Transformer Framework for Dense Prediction Tasks in Point Clouds [5.568628403034062]
  PCT-Prompt enhances standard Transformers by introducing a prompt-guided feature branch to improve performance in dense prediction tasks.<n>We show that PCT-Prompt significantly improves the adaptability of standard Transformers to dense prediction tasks, achieving strong performance in real-world scenarios.
  arXiv  Detail & Related papers  (2026-08-17T08:01:33Z)
• PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models [23.263895549689863]
  Existing methods rely solely on next-token prediction loss, using only language tokens for supervision.<n>mname explicitly supervises intermediate point cloud tokens to preserve fine-grained 3D geometric-semantic information.<n>Experiments on ModelNet40 and averse datasets demonstrate that our method achieves textbf2.08 pp improvement on average for classification tasks.
  arXiv  Detail & Related papers  (2026-02-28T02:17:46Z)
• PatchAlign3D: Local Feature Alignment for Dense 3D Shape understanding [67.15800065888887]
  Current foundation models for 3D shapes excel at global tasks (retrieval, classification) but transfer poorly to local part-level reasoning.<n>We introduce an encoder-only 3D model that produces language-aligned patch-level features directly from point clouds.<n>Our 3D encoder achieves zero-shot 3D part segmentation with fast single-pass inference without any test-time multi-view rendering.
  arXiv  Detail & Related papers  (2026-01-05T18:55:45Z)
• LitePT: Lighter Yet Stronger Point Transformer [50.6430530112838]
  We analyse the role of different computational blocks in 3D point cloud networks.<n>We propose a new, improved 3D point cloud backbone that employs convolutions in early stages and switches to attention for deeper layers.<n>The resulting LitePT model has $3.6times$ fewer parameters, runs $2times$ faster, and uses $2times$ less memory than the state-of-the-art Point Transformer V3.
  arXiv  Detail & Related papers  (2025-12-15T18:59:57Z)
• Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis [51.37795317716487]
  We propose the Adaptive Point-Prompt Tuning (APPT) method, which fine-tunes pre-trained models with a modest number of parameters.<n>We convert raw point clouds into point embeddings by aggregating local geometry to capture spatial features followed by linear layers.<n>To calibrate self-attention across source domains of any modality to 3D, we introduce a prompt generator that shares weights with the point embedding module.
  arXiv  Detail & Related papers  (2025-08-30T06:02:21Z)
• Fast3D: Accelerating 3D Multi-modal Large Language Models for Efficient 3D Scene Understanding [24.964149224068027]
  We propose Fast3D, a plug-and-play visual token pruning framework for 3D MLLMs.<n>Global Attention Prediction (GAP) learns to predict the global attention distributions of the target model, enabling efficient token importance estimation.<n>SAP, introduces dynamic token budgets through attention-based complexity assessment, automatically adjusting layer-wise pruning ratios.
  arXiv  Detail & Related papers  (2025-07-12T16:29:02Z)
• Parameter-efficient Prompt Learning for 3D Point Cloud Understanding [10.23165979353247]
  This paper presents a parameter-efficient prompt tuning method to adapt a large multi-modal model for 3D point cloud understanding. A PromptLearner module is devised to replace hand-crafted prompts with learnable contexts. A lightweight PointAdapter module is arranged near target tasks to enhance prompt tuning for 3D point cloud understanding.
  arXiv  Detail & Related papers  (2024-02-24T14:20:50Z)
• Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment [55.11291053011696]
  This work presents a framework for dealing with 3D scene understanding when the labeled scenes are quite limited.<n>To extract knowledge for novel categories from the pre-trained vision-language models, we propose a hierarchical feature-aligned pre-training and knowledge distillation strategy.<n>In the limited reconstruction case, our proposed approach, termed WS3D++, ranks 1st on the large-scale ScanNet benchmark.
  arXiv  Detail & Related papers  (2023-12-01T15:47:04Z)
• Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models [46.42092771753465]
  We introduce Point-PEFT, a novel framework for adapting point cloud pre-trained models with minimal learnable parameters. Specifically, for a pre-trained 3D model, we freeze most of its parameters, and only tune the newly added PEFT modules on downstream tasks.
  arXiv  Detail & Related papers  (2023-10-04T16:49:36Z)
• Approximated Prompt Tuning for Vision-Language Pre-trained Models [54.326232586461614]
  In vision-language pre-trained models, prompt tuning often requires a large number of learnable tokens to bridge the gap between the pre-training and downstream tasks. We propose a novel Approximated Prompt Tuning (APT) approach towards efficient VL transfer learning.
  arXiv  Detail & Related papers  (2023-06-27T05:43:47Z)
• Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives [28.138689389803034]
  Large language models (LLMs) have shown increasing power on various natural language processing (NLP) tasks. Black-box tuning has been proposed to address this problem by optimizing task-specific prompts without accessing the gradients and hidden representations. We describe BBT-RGB, a suite of straightforward and complementary techniques for enhancing the efficiency and performance of black-box optimization.
  arXiv  Detail & Related papers  (2023-05-14T07:33:59Z)
• Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models [64.49254199311137]
  We propose a novel Instance-aware Dynamic Prompt Tuning (IDPT) strategy for pre-trained point cloud models. The essence of IDPT is to develop a dynamic prompt generation module to perceive semantic prior features of each point cloud instance. In experiments, IDPT outperforms full fine-tuning in most tasks with a mere 7% of the trainable parameters.
  arXiv  Detail & Related papers  (2023-04-14T16:03:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.