DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup

• URL: http://arxiv.org/abs/2508.13560v2
• Date: Thu, 21 Aug 2025 02:08:06 GMT
• Title: DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup
• Authors: Zhen Qu, Xian Tao, Xinyi Gong, ShiChen Qu, Xiaopei Zhang, Xingang Wang, Fei Shen, Zhengtao Zhang, Mukesh Prasad, Guiguang Ding,
• Abstract summary: We propose DictAS, which enables a unified model to detect visual anomalies in unseen object categories without any retraining on the target data.<n>D DictAS mainly consists of three components: Dictionary Construction - to simulate the index and content of a real dictionary using features from normal reference images.<n>Experiments on seven public industrial and medical datasets demonstrate that DictAS consistently outperforms state-of-the-art FSAS methods.
• Score: 19.78332125963566
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent vision-language models (e.g., CLIP) have demonstrated remarkable class-generalizable ability to unseen classes in few-shot anomaly segmentation (FSAS), leveraging supervised prompt learning or fine-tuning on seen classes. However, their cross-category generalization largely depends on prior knowledge of real seen anomaly samples. In this paper, we propose a novel framework, namely DictAS, which enables a unified model to detect visual anomalies in unseen object categories without any retraining on the target data, only employing a few normal reference images as visual prompts. The insight behind DictAS is to transfer dictionary lookup capabilities to the FSAS task for unseen classes via self-supervised learning, instead of merely memorizing the normal and abnormal feature patterns from the training set. Specifically, DictAS mainly consists of three components: (1) Dictionary Construction - to simulate the index and content of a real dictionary using features from normal reference images. (2) Dictionary Lookup - to retrieve queried region features from the dictionary via a sparse lookup strategy. When a query feature cannot be retrieved, it is classified as an anomaly. (3) Query Discrimination Regularization - to enhance anomaly discrimination by making abnormal features harder to retrieve from the dictionary. To achieve this, Contrastive Query Constraint and Text Alignment Constraint are further proposed. Extensive experiments on seven public industrial and medical datasets demonstrate that DictAS consistently outperforms state-of-the-art FSAS methods.

Related papers

• Vocabulary-free Fine-grained Visual Recognition via Enriched Contextually Grounded Vision-Language Model [52.01031460230826]
  Traditional approaches rely heavily on fixed vocabularies and closed-set classification paradigms.<n>Recent research has demonstrated that combining large language models with vision-language models (VLMs) makes open-set recognition possible.<n>We propose our training-free method, Enriched-FineR, which demonstrates state-of-the-art results in fine-grained visual recognition.
  arXiv  Detail & Related papers  (2025-07-30T20:06:01Z)
• Test-time Vocabulary Adaptation for Language-driven Object Detection [42.25065847785535]
  We propose a plug-and-play Vocabulary Adapter (VocAda) to refine the user-defined vocabulary.<n>VocAda does not require any training, it operates at inference time in three steps.<n> Experiments on COCO and Objects365 with three state-of-the-art detectors show that VocAda consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-31T01:15:29Z)
• Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control [43.860799289234755]
  We propose a framework for evaluating feature dictionaries in the context of specific tasks, by comparing them against emphmagnitude feature dictionaries. First, we demonstrate that supervised dictionaries achieve excellent approximation, control, and interpretability of model computations on the task. We apply this framework to the indirect object identification (IOI) task using GPT-2 Small, with sparse autoencoders (SAEs) trained on either the IOI or OpenWebText datasets.
  arXiv  Detail & Related papers  (2024-05-14T07:07:13Z)
• Learning Interpretable Queries for Explainable Image Classification with Information Pursuit [16.192225229327242]
  Information Pursuit (IP) is an explainable prediction algorithm that greedily selects a sequence of interpretable queries about the data.<n>This paper introduces a novel approach: learning a dictionary of interpretable queries directly from the dataset.
  arXiv  Detail & Related papers  (2023-12-16T21:43:07Z)
• Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment [53.2701026843921]
  Large-scale pre-trained Vision Language Models (VLMs) have proven effective for zero-shot classification. In this paper, we aim at a more challenging setting, Realistic Zero-Shot Classification, which assumes no annotation but instead a broad vocabulary. We propose the Self Structural Semantic Alignment (S3A) framework, which extracts structural semantic information from unlabeled data while simultaneously self-learning.
  arXiv  Detail & Related papers  (2023-08-24T17:56:46Z)
• Multi-Modal Classifiers for Open-Vocabulary Object Detection [104.77331131447541]
  The goal of this paper is open-vocabulary object detection (OVOD) We adopt a standard two-stage object detector architecture. We explore three ways via: language descriptions, image exemplars, or a combination of the two.
  arXiv  Detail & Related papers  (2023-06-08T18:31:56Z)
• Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations [20.00016240535205]
  Most weakly supervised named entity recognition models rely on domain-specific dictionaries provided by experts. We present a novel framework, HighGEN, that generates NER datasets with high-coverage pseudo-dictionaries. We demonstrate that HighGEN outperforms the previous best model by an average F1 score of 4.7 across five NER benchmark datasets.
  arXiv  Detail & Related papers  (2022-10-14T07:36:44Z)
• DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection [118.36746273425354]
  This paper presents a paralleled visual-concept pre-training method for open-world detection by resorting to knowledge enrichment from a designed concept dictionary. By enriching the concepts with their descriptions, we explicitly build the relationships among various concepts to facilitate the open-domain learning. The proposed framework demonstrates strong zero-shot detection performances, e.g., on the LVIS dataset, our DetCLIP-T outperforms GLIP-T by 9.9% mAP and obtains a 13.5% improvement on rare categories.
  arXiv  Detail & Related papers  (2022-09-20T02:01:01Z)
• Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph [10.64488240379972]
  In cross-lingual text classification, it is required that task-specific training data in high-resource source languages are available. Collecting such training data can be infeasible because of the labeling cost, task characteristics, and privacy concerns. This paper proposes an alternative solution that uses only task-independent word embeddings of high-resource languages and bilingual dictionaries.
  arXiv  Detail & Related papers  (2021-09-09T16:40:40Z)
• MASKER: Masked Keyword Regularization for Reliable Text Classification [73.90326322794803]
  We propose a fine-tuning method, coined masked keyword regularization (MASKER), that facilitates context-based prediction. MASKER regularizes the model to reconstruct the keywords from the rest of the words and make low-confidence predictions without enough context. We demonstrate that MASKER improves OOD detection and cross-domain generalization without degrading classification accuracy.
  arXiv  Detail & Related papers  (2020-12-17T04:54:16Z)
• Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressions [87.33156149634392]
  We critically examine RefCOg, a standard benchmark for visual referring expression recognition. We show that 83.7% of test instances do not require reasoning on linguistic structure. We propose two methods, one based on contrastive learning and the other based on multi-task learning, to increase the robustness of ViLBERT.
  arXiv  Detail & Related papers  (2020-05-04T17:09:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.