Learning Generalizable Multimodal Representations for Software Vulnerability Detection

• URL: http://arxiv.org/abs/2604.25711v2
• Date: Thu, 30 Apr 2026 08:17:42 GMT
• Title: Learning Generalizable Multimodal Representations for Software Vulnerability Detection
• Abstract summary: We propose MultiVul, a framework that aligns code and comment representations through dual similarity learning and consistency regularization.<n>Experiments on widely adopted DiverseVul and Devign datasets show that MultiVul achieves up to 27.07% F1 improvement over prompting-based methods.
• Score: 25.453510500843688
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Source code and its accompanying comments are complementary yet naturally aligned modalities-code encodes structural logic while comments capture developer intent. However, existing vulnerability detection methods mostly rely on single-modality code representations, overlooking the complementary semantic information embedded in comments and thus limiting their generalization across complex code structures and logical relationships. To address this, we propose MultiVul, a multimodal contrastive framework that aligns code and comment representations through dual similarity learning and consistency regularization, augmented with diverse code-text pairs to improve robustness. Experiments on widely adopted DiverseVul and Devign datasets across four large language models (LLMs) (i.e., DeepSeek-Coder-6.7B, Qwen2.5-Coder-7B, StarCoder2-7B, and CodeLlama-7B) show that MultiVul achieves up to 27.07% F1 improvement over prompting-based methods and 13.37% over code-only Fine-Tuning, while maintaining comparable inference efficiency.

Related papers

• Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study [8.409799680956013]
  General-purpose code embeddings power tools for code search, classification, and retrieval.<n>We study an alternative: contrastive pretraining of small encoders with synthetically generated natural-language descriptions emphasizing code functionality and intent.
  arXiv  Detail & Related papers  (2026-09-03T11:41:09Z)
• Completion by Comprehension: Guiding Code Generation with Multi-Granularity Understanding [37.78627994991325]
  CoCo is a novel framework that enables code Completion by of multi-granularity context from large-scale code repositories.<n>Experiments on CrossCodeEval and RepoEval benchmarks demonstrate that CoCo consistently surpasses state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-12-04T07:37:59Z)
• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• Code-Craft: Hierarchical Graph-Based Code Summarization for Enhanced Context Retrieval [0.0]
  We present Hierarchical Code Graph Summarization (HCGS), a novel approach that constructs a multi-layered representation of a by generating structured summaries in a bottom-up fashion from a code graph.<n>HCGS consistently outperforms traditional code-only retrieval across all metrics.
  arXiv  Detail & Related papers  (2025-04-11T20:57:27Z)
• Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
  Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification. In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction. Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
  arXiv  Detail & Related papers  (2024-10-24T09:09:20Z)
• When simplicity meets effectiveness: Detecting code comments coherence with word embeddings and LSTM [6.417777780911223]
  Code comments play a crucial role in software development, as they provide programmers with practical information. Developers tend to leave comments unchanged after updating the code, resulting in a discrepancy between the two artifacts. It is crucial to identify if, given a code snippet, its corresponding comment is coherent and reflects well the intent behind the code.
  arXiv  Detail & Related papers  (2024-05-25T15:21:27Z)
• Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective [85.48043537327258]
  We propose MANGO (comMents As Natural loGic pivOts), including a comment contrastive training strategy and a corresponding logical comment decoding strategy. Results indicate that MANGO significantly improves the code pass rate based on the strong baselines. The robustness of the logical comment decoding strategy is notably higher than the Chain-of-thoughts prompting.
  arXiv  Detail & Related papers  (2024-04-11T08:30:46Z)
• Language Agnostic Code Embeddings [61.84835551549612]
  We focus on the cross-lingual capabilities of code embeddings across different programming languages. Code embeddings comprise two distinct components: one deeply tied to the nuances and syntax of a specific language, and the other remaining agnostic to these details. We show that when we isolate and eliminate this language-specific component, we witness significant improvements in downstream code retrieval tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:34:52Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• A Transformer-based Approach for Source Code Summarization [86.08359401867577]
  We learn code representation for summarization by modeling the pairwise relationship between code tokens. We show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin.
  arXiv  Detail & Related papers  (2020-05-01T23:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.