SAM3-LoRA: Parameter-Efficient Adaptation of a Concept-Promptable Foundation Model for Multi-Class Structural Defect Segmentation

• URL: http://arxiv.org/abs/2609.00469v1
• Date: Mon, 31 Aug 2026 23:19:59 GMT
• Title: SAM3-LoRA: Parameter-Efficient Adaptation of a Concept-Promptable Foundation Model for Multi-Class Structural Defect Segmentation
• Abstract summary: This study applies Low-Rank Adaptation (LoRA) to SAM3 for multi-class structural defect segmentation.<n>We describe a supervision procedure that trains a concept-promptable model directly from COCO-style class-labeled instance segmentation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Promptable segmentation foundation models such as SAM3 accept an open-vocabulary text concept and return every instance matching it, but adapting them to a specialized domain by full fine-tuning is computationally prohibitive for the organizations that would benefit most. This study applies Low-Rank Adaptation (LoRA) to SAM3 for multi-class structural defect segmentation and examines both how such a model can be supervised from conventional annotation and whether the resulting efficiency gain transfers across datasets. Two contributions are methodological. First, we describe a supervision procedure that trains a concept-promptable model directly from COCO-style class-labeled instance segmentation by using the category name itself as the prompt, requiring no prompt templates, no synonym expansion, and no learned class embeddings. Second, we identify and mitigate a failure mode specific to this setting: because a conventional annotation file yields positive prompts exclusively, the model's presence prediction decouples from the text condition and degenerates into responding to any prompt, a collapse that is invisible to every metric computed on positive prompts alone. Exhaustive hard-negative prompting, in which every dataset category absent from an image is issued as a zero-detection query, addresses this at no annotation cost. Two adapter placements were compared under an identical protocol, updating 0.121% and 1.341% of model parameters. On a purpose-built tunnel lining dataset, pixel intersection-over-union improved from 0.017 to 0.338 and instance-level recall from 0.375 to 0.672; on the independent public Structural Defects Dataset, from 0.017 to 0.855 and from 0.574 to 1.000. Improvements were directionally consistent across ten metrics on both datasets, and the largest per-category gains occurred precisely where zero-shot competence was absent.

Related papers

• A Unified Backbone--Expert Framework with Relation-Token and Residual--Classifier Interfaces for Automatic Modulation Recognition [1.547549252134621]
  We propose a unified backbone-expert framework with a common convolutional state-space backbone and two specialized interfaces.<n>Our framework achieves overall average accuracies of 67.28 pm 0.14% on RML2016.10b and 87.19 pm 0.77% on HisarMod 2019.
  arXiv  Detail & Related papers  (2026-08-15T10:26:42Z)
• Mapping and Measuring the Behavioral Evolution of Large Language Models [28.603443859513153]
  Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations.<n>We characterize the output behavior of 32 models from six families using their responses to a shared bank of 10,000 prompts.
  arXiv  Detail & Related papers  (2026-08-11T15:07:28Z)
• EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass [79.69317943751223]
  We propose an Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM)<n>EOVSAM adapts SAM 3 for single-pass prediction.<n>It consistently improves segmentation accuracy over vanilla SAM 3 on all evaluated datasets.
  arXiv  Detail & Related papers  (2026-08-03T14:15:37Z)
• SPARE: Structural Parameter-Free Affinity Regularization for Flow Matching [30.43836162456455]
  Regularizing internal representations has emerged as an effective accelerator, yet existing methods split into two families with complementary costs.<n>We propose Structural.<n>free Affinity Regularization (SPARE), a regularizer that matches the pairwise affinities of intermediate tokens to those of the clean latents.<n>On ImageNet $256 times 256$ with SiT backbones under matched 400K-iteration budgets, SPARE adds no encoder, head, or parameters and only 0.08 GB of training memory.
  arXiv  Detail & Related papers  (2026-08-03T09:50:51Z)
• Closing the Modality Gap in Zero-Shot HAR: Contrastive Training and Separability-Optimized Prototypes on IMU Data [0.0]
  We evaluate seven configurations combining three inference methods with two training pipelines on the PAMAP2 dataset.<n>We find that the modality gap is a training-time phenomenon governed by the objective.<n>We also demonstrate that overall accuracy is a misleading primary metric when test-set class distributions are imbalanced.
  arXiv  Detail & Related papers  (2026-06-09T12:39:41Z)
• Architecture-Aware Explanation Auditing for Industrial Visual Inspection [1.3054033103300278]
  This paper operationalizes an architecture-aware explanation audit protocol grounded in the native-readout hypothesis.<n>On WM-811K wafer maps (9 classes, 172k images) under a three-seed zero-fill perturbation protocol, ViT-Tiny + Attention Rollout attains Deletion AUC 0.211 against 0.432-0.525 for Swin-Tiny.
  arXiv  Detail & Related papers  (2026-05-14T01:48:00Z)
• A Fixed-Point Approach to Unified Prompt-Based Counting [51.20608895374113]
  This paper aims to establish a comprehensive prompt-based counting framework capable of generating density maps for objects indicated by various prompt types, such as box, point, and text. Our model excels in prominent class-agnostic datasets and exhibits superior performance in cross-dataset adaptation tasks.
  arXiv  Detail & Related papers  (2024-03-15T12:05:44Z)
• Generalized Category Discovery with Clustering Assignment Consistency [56.92546133591019]
  Generalized category discovery (GCD) is a recently proposed open-world task. We propose a co-training-based framework that encourages clustering consistency. Our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2023-10-30T00:32:47Z)
• Prompt Tuning for Parameter-efficient Medical Image Segmentation [79.09285179181225]
  We propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. We pre-train this architecture with a dedicated dense self-supervision scheme based on assignments to online generated prototypes. We demonstrate that the resulting neural network model is able to attenuate the gap between fully fine-tuned and parameter-efficiently adapted models.
  arXiv  Detail & Related papers  (2022-11-16T21:55:05Z)
• Label-Free Model Evaluation with Semi-Structured Dataset Representations [78.54590197704088]
  Label-free model evaluation, or AutoEval, estimates model accuracy on unlabeled test sets. In the absence of image labels, based on dataset representations, we estimate model performance for AutoEval with regression. We propose a new semi-structured dataset representation that is manageable for regression learning while containing rich information for AutoEval.
  arXiv  Detail & Related papers  (2021-12-01T18:15:58Z)
• Federated Unsupervised Representation Learning [56.715917111878106]
  We formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision. FedCA is composed of two key modules: dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and alignment module to align the representation of each client on a base model trained on a public data.
  arXiv  Detail & Related papers  (2020-10-18T13:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.