RLBind: Adversarial-Invariant Cross-Modal Alignment for Unified Robust Embeddings

• URL: http://arxiv.org/abs/2509.14383v1
• Date: Wed, 17 Sep 2025 19:35:52 GMT
• Title: RLBind: Adversarial-Invariant Cross-Modal Alignment for Unified Robust Embeddings
• Abstract summary: Unified multi-modal encoders that bind vision, audio, and other sensors into a shared embedding space are attractive building blocks for robot perception and decision-making.<n>We introduce RLBind, a two-stage adversarial-invariant cross-modal alignment framework for robust unified embeddings.<n> RLBind consistently outperforms the LanguageBind backbone and standard fine-tuning baselines in both clean accuracy and norm-bounded adversarial robustness.
• Score: 0.47097887983573417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified multi-modal encoders that bind vision, audio, and other sensors into a shared embedding space are attractive building blocks for robot perception and decision-making. However, on-robot deployment exposes the vision branch to adversarial and natural corruptions, making robustness a prerequisite for safety. Prior defenses typically align clean and adversarial features within CLIP-style encoders and overlook broader cross-modal correspondence, yielding modest gains and often degrading zero-shot transfer. We introduce RLBind, a two-stage adversarial-invariant cross-modal alignment framework for robust unified embeddings. Stage 1 performs unsupervised fine-tuning on clean-adversarial pairs to harden the visual encoder. Stage 2 leverages cross-modal correspondence by minimizing the discrepancy between clean/adversarial features and a text anchor, while enforcing class-wise distributional alignment across modalities. Extensive experiments on Image, Audio, Thermal, and Video data show that RLBind consistently outperforms the LanguageBind backbone and standard fine-tuning baselines in both clean accuracy and norm-bounded adversarial robustness. By improving resilience without sacrificing generalization, RLBind provides a practical path toward safer multi-sensor perception stacks for embodied robots in navigation, manipulation, and other autonomy settings.

Related papers

• CrACK: Adversarial Attacks on Cross-Model Consistency in Collaborative Vision Foundation Models [16.64134816858303]
  Training-free collaborative pipelines that integrate Vision Foundation Models such as CLIP, SAM, and DINO achieve strong open-vocabulary dense prediction.<n>We propose CrACK, an inference-time attack that exploits this interface without modifying any input pixel, model weight, or training data.<n>Experiments show that CrACK causes catastrophic degradation while every individual model continues to produce its unchanged standalone output.
  arXiv  Detail & Related papers  (2026-09-07T13:47:33Z)
• AdROD: HyperNetwork-based Adversarially Robust Object Detection for Autonomous Driving [4.4758385009686945]
  Camera-based object detectors are vulnerable to physical adversarial attacks designed to suppress detections.<n>This paper presents AdROD, an embedded, ensemble defense software designed for autonomous driving.
  arXiv  Detail & Related papers  (2026-08-17T02:52:01Z)
• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model [73.59026525350808]
  We propose a dual adversarial fine-tuning framework that jointly optimize visual and semantic supervision signals from two modalities.<n>Our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering tasks.
  arXiv  Detail & Related papers  (2026-07-21T10:49:29Z)
• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping [61.459927600301654]
  Multi-condition control is bottlenecked by the conventional concatenate-and-attend'' strategy.<n>Our analysis reveals that much of this cross-modal interaction is spatially or semantically redundant.<n>We propose Position-aligned and Keyword-scoped Attention (PKA), a highly efficient framework designed to eliminate these redundancies.
  arXiv  Detail & Related papers  (2026-02-06T16:39:10Z)
• OmniDrive-R1: Reinforcement-driven Interleaved Multi-modal Chain-of-Thought for Trustworthy Vision-Language Autonomous Driving [11.793091014248025]
  We introduce OmniDrive-R1, an end-to-end VLM framework designed for autonomous driving.<n>Our core innovation is an Reinforcement-driven visual grounding capability.<n>This capability is enabled by our pure two-stage reinforcement learning training pipeline and Clip-GRPO algorithm.
  arXiv  Detail & Related papers  (2025-12-16T03:19:28Z)
• $\mathcal{E}_0$: Enhancing Generalization and Fine-Grained Control in VLA Models via Continuized Discrete Diffusion [65.77755100137728]
  We introduce E0, a continuized discrete diffusion framework that formulates action generation as iterative denoising over quantized action tokens.<n>E0 achieves state-of-the-art performance across 14 diverse environments, outperforming strong baselines by 10.7% on average.
  arXiv  Detail & Related papers  (2025-11-26T16:14:20Z)
• When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models [81.7618160628979]
  Vision-Language-Action (VLA) models are vulnerable to adversarial attacks, yet universal and transferable attacks remain underexplored.<n>We introduce UPA-RFAS (Universal Patch Attack via Robust Feature, Attention, and Semantics), a unified framework that learns a single physical patch in a shared feature space.<n> Experiments across diverse VLA models, manipulation suites, and physical executions show that UPA-RFAS consistently transfers across models, tasks, and viewpoints.
  arXiv  Detail & Related papers  (2025-11-26T09:16:32Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• DROID: Dual Representation for Out-of-Scope Intent Detection [4.768906732056304]
  DROID is a compact framework that combines two complementary encoders -- the Universal Sentence Detection (USE) for broad semantic generalization and the Transformer-based Denoising Autoencoder (TSDAE) for domain-specific contextual distinctions.<n>Our results show that dual-encoder representations with simple calibration can yield robust, scalable, and reliable OOS detection for neural dialogue systems.
  arXiv  Detail & Related papers  (2025-10-15T21:29:52Z)
• DRCP: Diffusion on Reinforced Cooperative Perception for Perceiving Beyond Limits [11.34052678290095]
  Diffusion on Reinforced Cooperative Perception (DRCP) is a real-time deployable framework designed to address issues in dynamic driving environments.<n>The proposed system achieves real-time performance on mobile platforms while significantly improving robustness under challenging conditions.
  arXiv  Detail & Related papers  (2025-09-29T15:13:03Z)
• QuadKAN: KAN-Enhanced Quadruped Motion Control via End-to-End Reinforcement Learning [2.26464274357758]
  We address vision-guided quadruped motion control with reinforcement learning (RL)<n>We propose QuadKAN, a cross-modal policy instantiated with Kolmogorov-Arnold Networks (KANs)<n>We show that QuadKAN achieves consistently higher returns, greater distances, and fewer collisions than state-of-the-art (SOTA) baselines.
  arXiv  Detail & Related papers  (2025-08-26T16:05:32Z)
• SCING:Towards More Efficient and Robust Person Re-Identification through Selective Cross-modal Prompt Tuning [17.104905795008555]
  We propose a framework named Selective Cross-modal Prompt Tuning (SCING) that enhances cross-modal alignment and robustness against real-worlds.<n>Our framework eliminates heavy adapters while maintaining efficient inference, achieving an optimal trade-off between performance and computational overhead.
  arXiv  Detail & Related papers  (2025-07-01T07:21:31Z)
• Adversarial Robustness for Unified Multi-Modal Encoders via Efficient Calibration [12.763688592842717]
  We present the first comprehensive study of adversarial vulnerability in unified multi-modal encoders.<n>Non-visual inputs, such as audio and point clouds, are especially fragile.<n>Our method improves adversarial robustness by up to 47.3 percent at epsilon = 4/255.
  arXiv  Detail & Related papers  (2025-05-17T08:26:04Z)
• Deep Positive-Negative Prototypes for Adversarially Robust Discriminative Prototypical Learning [0.24999074238880484]
  We propose a novel framework named Adversarially trained Deep Positive-Negative Prototypes (Adv-DPNP)<n>Adv-DPNP integrates discriminative prototype-based learning with adversarial training.<n>We show that Adv-DPNP achieves the highest average accuracy across severities and corruption types.
  arXiv  Detail & Related papers  (2025-04-03T15:42:58Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [63.87313550399871]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose Self-supervised Transfer (PST) and FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• CAFuser: Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenes [56.52618054240197]
  We propose a novel, condition-aware multimodal fusion approach for robust semantic perception of driving scenes.<n>Our method, CAFuser, uses an RGB camera input to classify environmental conditions and generate a Condition Token.<n>Our model significantly improves robustness and accuracy, especially in adverse-condition scenarios.
  arXiv  Detail & Related papers  (2024-10-14T17:56:20Z)
• Unpaired Adversarial Learning for Single Image Deraining with Rain-Space Contrastive Constraints [61.40893559933964]
  We develop an effective unpaired SID method which explores mutual properties of the unpaired exemplars by a contrastive learning manner in a GAN framework, named as CDR-GAN. Our method performs favorably against existing unpaired deraining approaches on both synthetic and real-world datasets, even outperforms several fully-supervised or semi-supervised models.
  arXiv  Detail & Related papers  (2021-09-07T10:00:45Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.