Res$^2$CLIP: Few-Shot Generalist Anomaly Detection with Residual-to-Residual Alignment

• URL: http://arxiv.org/abs/2605.16171v1
• Date: Fri, 15 May 2026 16:49:58 GMT
• Title: Res$^2$CLIP: Few-Shot Generalist Anomaly Detection with Residual-to-Residual Alignment
• Abstract summary: Few-shot Generalist Anomaly Detection requires models to generalize to novel categories without retraining.<n>Existing CLIP-based methods face two major challenges: coarse-grained unified text prompts struggle to adapt to fine-grained foreground-background differences.<n>We propose to shift multimodal alignment entirely into a unified residual space, where residual representations naturally eliminate fine-grained normal feature differences.
• Score: 21.61732087909472
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot Generalist Anomaly Detection requires models to generalize to novel categories without retraining, posing significant challenges in real-world scenarios with scarce samples and rapidly changing categories. Existing CLIP-based methods face two major challenges: coarse-grained unified text prompts struggle to adapt to fine-grained foreground-background differences, causing cross-granularity mismatch; and fine-tuning on auxiliary datasets disrupts CLIP's inherent open-world generalization due to domain shift, leading to cross-category generalization degradation. To address these, we propose to shift multimodal alignment entirely into a unified residual space, where residual representations naturally eliminate fine-grained normal feature differences across regions and class-specific biases, simultaneously resolving both problems. Based on this insight, Res$^2$CLIP, the first residual-to-residual alignment framework that symmetrically bridges visual and text modalities within CLIP's residual space, is designed. The framework is developed from a residual perspective into three branches: a text prompt-based branch, a visual prompt-based branch, and a novel residual-to-residual alignment branch. All learnable optimizations are constrained within the residual domain, and the residual alignment optimization objectives are designed to force the model to focus on relative anomaly deviations rather than optimizing class-specific features. Experiments on multiple datasets demonstrate the effectiveness of our architecture. The code is available at https://github.com/hito2448/Res2CLIP.

Related papers

• Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection [7.184018933441995]
  Vision-language models offer a promising approach for zero-shot anomaly detection (ZSAD)<n>Normal and anomalous text prototypes exhibit a high semantic overlap.<n>We propose Proximity-CLIP, a framework that visually calibrates the semantic margin to guide visual adaptation.
  arXiv  Detail & Related papers  (2026-09-07T08:48:27Z)
• FAIR: Feature-Augmented Implicit Regularization for AI-generated Fake Image Detection [5.673956549751241]
  Generalization remains a critical bottleneck in AI-generated image detection.<n>We propose Feature-Augmented Implicit Regularization (FAIR)<n>We show that FAIR boosts accuracy by up to 8.04% and establishes new state-of-the-art robustness in zero-shot transfer scenarios.
  arXiv  Detail & Related papers  (2026-07-24T08:34:56Z)
• ATRS: Adaptive Trajectory Re-splitting via a Shared Neural Policy for Parallel Optimization [14.810654249835572]
  We propose ATRS, a novel framework that embeds a shared Deep Reinforcement Learning policy into the parallel ADMM loop.<n>We show that ATRS convergence accelerates by up to 26.2% and the computation time by up to 19.1%.<n>Real-world experiments further confirm its applicability to both large-scale offline global planning and real-time onboard replanning within 35 ms per cycle.
  arXiv  Detail & Related papers  (2026-04-24T16:58:14Z)
• From Global to Local: Rethinking CLIP Feature Aggregation for Person Re-Identification [20.74226476678832]
  We propose SAGA-ReID, which reconstructs identity representations by aligning intermediate patch tokens with anchor vectors parameterized in CLIP's text embedding space.<n> Controlled experiments isolate the aggregation mechanism under two qualitatively distinct conditions.<n> Benchmark evaluations confirm consistent gains over CLIP-ReID across standard and occluded settings.
  arXiv  Detail & Related papers  (2026-04-24T03:37:21Z)
• Semantic-based Distributed Learning for Diverse and Discriminative Representations [26.214742067718277]
  In large-scale distributed scenarios, increasingly complex tasks demand more intelligent collaboration across networks.<n>We propose a novel distributed learning framework that ensures both diverse and discriminative representations.<n>In both cases, the resulting optimal solutions are theoretically proven to maintain discriminative and diverse properties.
  arXiv  Detail & Related papers  (2026-04-20T13:22:58Z)
• ReHARK: Refined Hybrid Adaptive RBF Kernels for Robust One-Shot Vision-Language Adaptation [3.8707695363745214]
  ReHARK is a training-free framework that reinterprets few-shot adaptation through global proximal regularization.<n>A new state-of-the-art for one-shot adaptation is established by ReHARK, which achieves an average accuracy of 65.83%.
  arXiv  Detail & Related papers  (2026-03-12T04:59:09Z)
• HAAF: Hierarchical Adaptation and Alignment of Foundation Models for Few-Shot Pathology Anomaly Detection [10.649984141835189]
  We propose the Hierarchical Adaptation and Alignment Framework (HAAF)<n>At its core is a novel Cross-Level Scaled Alignment mechanism that enforces a sequential calibration order.<n>A dual-branch inference strategy integrates semantic scores with geometric prototypes to ensure stability in few-shot settings.
  arXiv  Detail & Related papers  (2026-01-24T10:31:21Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation [104.59740403500132]
  Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance.<n>We propose a unified modality-relax segmentation network (UniMRSeg) through hierarchical self-supervised compensation (HSSC)<n>Our approach hierarchically bridges representation gaps between complete and incomplete modalities across input, feature and output levels.
  arXiv  Detail & Related papers  (2025-09-19T17:29:25Z)
• Manifold-aware Representation Learning for Degradation-agnostic Image Restoration [135.90908995927194]
  Image Restoration (IR) aims to recover high quality images from degraded inputs affected by various corruptions such as noise, blur, haze, rain, and low light conditions.<n>We present MIRAGE, a unified framework for all in one IR that explicitly decomposes the input feature space into three semantically aligned parallel branches.<n>This modular decomposition significantly improves generalization and efficiency across diverse degradations.
  arXiv  Detail & Related papers  (2025-05-24T12:52:10Z)
• Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration [109.38288333994407]
  Contrastive Prompt Learning (CPL) is a novel framework that fundamentally enhances prompt-task alignment.<n>Our framework establishes new state-of-the-art performance while maintaining parameter efficiency, offering a principled solution for unified image restoration.
  arXiv  Detail & Related papers  (2025-04-14T08:24:57Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.<n>We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.