Occlusion Robustness of CLIP for Military Vehicle Classification

• URL: http://arxiv.org/abs/2508.20760v2
• Date: Tue, 02 Sep 2025 15:53:42 GMT
• Title: Occlusion Robustness of CLIP for Military Vehicle Classification
• Abstract summary: Vision-language models (VLMs) like CLIP enable zero-shot classification by aligning images and text in a shared embedding space.<n>We investigate CLIP variants' robustness to occlusion using a custom dataset of 18 military vehicle classes.<n>Four key insights emerge: Transformer-based CLIP models consistently outperform CNNs.
• Score: 0.8110981870695059
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) like CLIP enable zero-shot classification by aligning images and text in a shared embedding space, offering advantages for defense applications with scarce labeled data. However, CLIP's robustness in challenging military environments, with partial occlusion and degraded signal-to-noise ratio (SNR), remains underexplored. We investigate CLIP variants' robustness to occlusion using a custom dataset of 18 military vehicle classes and evaluate using Normalized Area Under the Curve (NAUC) across occlusion percentages. Four key insights emerge: (1) Transformer-based CLIP models consistently outperform CNNs, (2) fine-grained, dispersed occlusions degrade performance more than larger contiguous occlusions, (3) despite improved accuracy, performance of linear-probed models sharply drops at around 35% occlusion, (4) by finetuning the model's backbone, this performance drop occurs at more than 60% occlusion. These results underscore the importance of occlusion-specific augmentations during training and the need for further exploration into patch-level sensitivity and architectural resilience for real-world deployment of CLIP.

Related papers

• OSCS-SupCon: Orthogonal Sigmoid-based Common and Style Supervised Contrastive Learning for Robust Feature Disentanglement [13.983602516442454]
  Supervised Contrastive Learning (SupCon) has achieved strong performance by explicitly modeling pairwise relationships among samples.<n>Existing SupCon-based methods suffer from two key limitations: negative-sample dilution induced by the standard InfoNCE loss, and feature-space entanglement.<n>We propose OSCS-SupCon (Orthogonal Sigmoid-based Common and Style Supervised Contrastive Learning) to address these issues.
  arXiv  Detail & Related papers  (2026-05-29T06:37:20Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• OVS-DINO: Open-Vocabulary Segmentation via Structure-Aligned SAM-DINO with Language Guidance [53.45125687428705]
  Open-Vocabulary aims to segment image regions beyond predefined category sets by leveraging semantic descriptions.<n>Recent efforts have incorporated Vision Foundation Models (VFMs) like DINO to alleviate these limitations.<n>We propose OVS-DINO, a novel framework that revitalizes latent edge-sensitivity of DINO through structural alignment.
  arXiv  Detail & Related papers  (2026-04-09T16:57:11Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• Conformal Lesion Segmentation for 3D Medical Images [82.92159832699583]
  We propose a risk-constrained framework that calibrates data-driven thresholds via conformalization to ensure the test-time FNR remains below a target tolerance.<n>We validate the statistical soundness and predictive performance of CLS on six 3D-LS datasets across five backbone models, and conclude with actionable insights for deploying risk-aware segmentation in clinical practice.
  arXiv  Detail & Related papers  (2025-10-19T08:21:00Z)
• Hierarchical Self-Supervised Representation Learning for Depression Detection from Speech [51.14752758616364]
  Speech-based depression detection (SDD) is a promising, non-invasive alternative to traditional clinical assessments.<n>We propose HAREN-CTC, a novel architecture that integrates multi-layer SSL features using cross-attention within a multitask learning framework.<n>The model achieves state-of-the-art macro F1-scores of 0.81 on DAIC-WOZ and 0.82 on MODMA, outperforming prior methods across both evaluation scenarios.
  arXiv  Detail & Related papers  (2025-10-05T09:32:12Z)
• Trade-offs in Cross-Domain Generalization of Foundation Model Fine-Tuned for Biometric Applications [18.08946802592489]
  Foundation models such as CLIP have demonstrated exceptional zero- and few-shot transfer capabilities across diverse vision tasks.<n>However, fine-tuned for highly specialized biometric tasks, face recognition (FR), morphing attack detection (MAD), and presentation attack detection (PAD) may suffer from over-specialization.<n>We systematically quantify these trade-offs by evaluating three instances of CLIP fine-tuned for FR, MAD, and PAD.
  arXiv  Detail & Related papers  (2025-09-18T12:58:18Z)
• Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception [71.26728044621458]
  DeCLIP is a novel framework that enhances CLIP by decoupling the self-attention module to obtain content'' and context'' features respectively.<n>It consistently achieves state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.
  arXiv  Detail & Related papers  (2025-08-15T06:43:51Z)
• LeakyCLIP: Extracting Training Data from CLIP [23.702455444224995]
  textbfLeakyCLIP aims to achieve high-quality, semantically accurate image reconstruction from CLIP embeddings.<n>We identify three key challenges in CLIP inversion: 1) non-robust features, 2) limited visual semantics in text embeddings, and 3) low reconstruction fidelity.
  arXiv  Detail & Related papers  (2025-08-01T16:32:48Z)
• Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning [48.11265601808718]
  We show that standard self-supervised contrastive learning objectives implicitly approximate a supervised variant we call the negatives-only supervised contrastive loss (NSCL)<n>We prove that the gap between the CL and NSCL losses vanishes as the number of semantic classes increases, under a bound that is both label-agnostic and architecture-independent.
  arXiv  Detail & Related papers  (2025-06-04T19:43:36Z)
• Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentation [19.749490092520006]
  Self-Calibrated CLIP (SC-CLIP) is a training-free method that calibrates CLIP to produce finer representations.<n>SC-CLIP boosts the performance of vanilla CLIP ViT-L/14 by 6.8 times.
  arXiv  Detail & Related papers  (2024-11-24T15:14:05Z)
• Affinity-Graph-Guided Contractive Learning for Pretext-Free Medical Image Segmentation with Minimal Annotation [55.325956390997]
  This paper proposes an affinity-graph-guided semi-supervised contrastive learning framework (Semi-AGCL) for medical image segmentation. The framework first designs an average-patch-entropy-driven inter-patch sampling method, which can provide a robust initial feature space. With merely 10% of the complete annotation set, our model approaches the accuracy of the fully annotated baseline, manifesting a marginal deviation of only 2.52%.
  arXiv  Detail & Related papers  (2024-10-14T10:44:47Z)
• Investigating the Semantic Robustness of CLIP-based Zero-Shot Anomaly Segmentation [2.722220619798093]
  We investigate the performance of a zero-shot anomaly segmentation algorithm by perturbing test data using three semantic transformations. We find that performance is consistently lowered on three CLIP backbones, regardless of model architecture or learning objective.
  arXiv  Detail & Related papers  (2024-05-13T17:47:08Z)
• Latent Enhancing AutoEncoder for Occluded Image Classification [2.6217304977339473]
  We introduce LEARN: Latent Enhancing feAture Reconstruction Network. An auto-encoder based network that can be incorporated into the classification model before its head. On the OccludedPASCAL3D+ dataset, the proposed LEARN outperforms standard classification models.
  arXiv  Detail & Related papers  (2024-02-10T12:22:31Z)
• FILP-3D: Enhancing 3D Few-shot Class-incremental Learning with Pre-trained Vision-Language Models [59.13757801286343]
  Few-shot class-incremental learning aims to mitigate the catastrophic forgetting issue when a model is incrementally trained on limited data.<n>We introduce the FILP-3D framework with two novel components: the Redundant Feature Eliminator (RFE) for feature space misalignment and the Spatial Noise Compensator (SNC) for significant noise.
  arXiv  Detail & Related papers  (2023-12-28T14:52:07Z)
• Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data [122.282521548393]
  Contrastive Language-Image Pre-training (CLIP) has become the standard for cross-modal image-text representation learning.<n>We introduce HELIP, a cost-effective strategy that improves CLIP models by exploiting challenging text-image pairs within existing datasets in continuous training.
  arXiv  Detail & Related papers  (2023-05-09T07:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.