AGC: Adaptive Geodesic Correction for Adversarial Robustness on Vision-Language Models

• URL: http://arxiv.org/abs/2605.15584v1
• Date: Fri, 15 May 2026 03:48:22 GMT
• Title: AGC: Adaptive Geodesic Correction for Adversarial Robustness on Vision-Language Models
• Abstract summary: AGC achieves superior performance across eight fine-grained datasets and three CLIP backbones.<n>Our findings reveal a fundamental geometric property of CLIP features, offering a highly efficient and effective paradigm for robust multimodal deployment.
• Score: 38.50355675955809
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models like CLIP have demonstrated remarkable zero-shot transfer capabilities. However, their susceptibility to imperceptible adversarial perturbations remains a critical security concern. While test-time defenses offer a pragmatic solution for deployed models, existing approaches typically rely on gradient-based optimization during inference, incurring significant computational overhead. In this paper, we revisit the role of data augmentation in CLIP robustness and observe that augmentations are not equally effective: specific augmentations consistently provide robust geometric cues that align with correct class semantics in the hyperspherical feature space. Based on this, we propose Adaptive Geodesic Correction (AGC), a training-free defense mechanism that requires no parameter updates. AGC identifies a reliable augmentation as a geometric anchor and corrects the input feature towards it, utilizing an adaptive step size to balance robustness against clean accuracy preservation. AGC achieves superior performance across eight fine-grained datasets and three CLIP backbones, improving average robust accuracy by 44.4\% over state-of-the-art baseline while delivering a 10$\times$ reduction in inference latency. Our findings reveal a fundamental geometric property of CLIP features, offering a highly efficient and effective paradigm for robust multimodal deployment.

Related papers

• CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting [11.613315225188302]
  We introduce Conformal Prediction (CP) and propose Conformalized Agentic Search (CAS)<n>This framework establishes reliability guarantees on both the retrieval and training sides.<n>Our framework significantly improves reasoning accuracy while drastically reducing redundant tool invocations.
  arXiv  Detail & Related papers  (2026-08-21T06:29:52Z)
• ReACT-CLIP: Response-Aware Test-Time Defense for Vision--Language Models [34.88445604400548]
  ReACT-CLIP is a test-time defense that determines how strongly each input should be corrected and whether defensive intervention is necessary.<n>It delivers substantial robustness gains across diverse attack types and strengths while largely preserving clean accuracy.
  arXiv  Detail & Related papers  (2026-08-02T08:06:45Z)
• Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards [73.44333771806282]
  Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as an effective paradigm for improving the reasoning capabilities of large language models.<n>We propose Correction-Oriented Policy Optimization (CIPO), a simple and effective extension to RLVR that converts on-policy failed trajectories into correction-oriented supervision.<n>CIPO improves learning effectiveness while explicitly enhancing the model's ability to correct its own errors.
  arXiv  Detail & Related papers  (2026-05-14T08:22:21Z)
• The Geometry of Robustness: Optimizing Loss Landscape Curvature and Feature Manifold Alignment for Robust Finetuning of Vision-Language Models [33.10473765967503]
  Generalization-preserving methods retain ID/OOD performance but leave models vulnerable to adversarial attacks.<n>Our key insight is that the robustness trade-off stems from two geometric failures: sharp, anisotropic minima in parameter space and unstable feature representations that deform under perturbation.<n>We propose GRACE, a unified fine-tuning framework that jointly regularizes the parameter-space curvature and feature-space invariance for VLMs.
  arXiv  Detail & Related papers  (2026-03-28T05:22:00Z)
• ReHARK: Refined Hybrid Adaptive RBF Kernels for Robust One-Shot Vision-Language Adaptation [3.8707695363745214]
  ReHARK is a training-free framework that reinterprets few-shot adaptation through global proximal regularization.<n>A new state-of-the-art for one-shot adaptation is established by ReHARK, which achieves an average accuracy of 65.83%.
  arXiv  Detail & Related papers  (2026-03-12T04:59:09Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• EAGER: Edge-Aligned LLM Defense for Robust, Efficient, and Accurate Cybersecurity Question Answering [10.78145758065258]
  EAGER integrates parameter-efficient quantization with domain-specific preference alignment to jointly optimize efficiency, robustness, and accuracy.<n> Experiments show that EAGER reduces adversarial attack success rates by up to 7.3x and improves QA accuracy by up to 55% over state-of-the-art defenses.
  arXiv  Detail & Related papers  (2025-11-24T06:49:48Z)
• DARN: Dynamic Adaptive Regularization Networks for Efficient and Robust Foundation Model Adaptation [0.0]
  We introduce Dynamic Adaptive Regularization Networks (DARN)<n>DARN integrates three key innovations: a lightweight Task Complexity Predictor ( TCP) that estimates per-sample difficulty, Adaptive Dropout Modulation (ADM) and Dynamic Capacity Gating (DCG)<n>In full fine-tuning (unfrozen backbone), DARN achieves a new state-of-the-art on the multi-task GeoBench benchmark (86.66% mIoU, +5.56 pp over prior SOTA).<n>In efficient adaptation (frozen backbone), DARN achieves SOTA-competitive accuracy (90.5% mIoU on Sen
  arXiv  Detail & Related papers  (2025-11-06T19:36:49Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation [93.38604803625294]
  We present UncertaintyRAG, a novel approach for long-context Retrieval-Augmented Generation (RAG) We use Signal-to-Noise Ratio (SNR)-based span uncertainty to estimate similarity between text chunks. UncertaintyRAG outperforms baselines by 2.03% on LLaMA-2-7B, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-10-03T17:39:38Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• $α$-OCC: Uncertainty-Aware Camera-based 3D Semantic Occupancy Prediction [32.78977564877008]
  Camera-based 3D Semantic Occupancy Prediction (OCC) aims to infer scene geometry and semantics from limited observations.<n>We first introduce Depth-UP, an uncertainty propagation framework that improves geometry completion by up to 11.58%.<n>For uncertainty (UQ), we propose the hierarchical conformal prediction (HCP) method, effectively handling the high-level class imbalance in OCC datasets.
  arXiv  Detail & Related papers  (2024-06-16T17:27:45Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.