Standing on the Shoulders of Giants: Stabilized Knowledge Distillation for Cross--Language Code Clone Detection

• URL: http://arxiv.org/abs/2605.02860v1
• Date: Mon, 04 May 2026 17:37:16 GMT
• Title: Standing on the Shoulders of Giants: Stabilized Knowledge Distillation for Cross--Language Code Clone Detection
• Abstract summary: Cross-language code clone detection (X-CCD) is challenging because semantically equivalent programs written in different languages often share little surface similarity.<n>We propose a knowledge distillation framework that transfers reasoning capabilities from DeepSeek-R1 into compact open-source student models for X-CCD detection.
• Score: 3.2116198597240846
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-language code clone detection (X-CCD) is challenging because semantically equivalent programs written in different languages often share little surface similarity. Although large language models (LLMs) have shown promise for semantic clone detection, their use as black-box systems raises concerns about cost, reproducibility, privacy, and unreliable output formatting. In particular, compact open-source models often struggle to follow reasoning-oriented prompts and to produce outputs that can be consistently mapped to binary clone labels. To address these limitations, we propose a knowledge distillation framework that transfers reasoning capabilities from DeepSeek-R1 into compact open-source student models for X-CCD. Using cross-language code pairs derived from Project CodeNet, we construct reasoning-oriented synthetic training data and fine-tune Phi3 and Qwen-Coder with LoRA adapters. We further introduce response stabilization methods, including forced conclusion prompting, a binary classification head, and a contrastive classification head, and evaluate model behavior using both predictive metrics and response rate. Experiments on Python--Java, Rust--Java, Rust--Python, and Rust--Ruby show that knowledge distillation consistently improves the reliability of compact models and often improves predictive performance, especially under distribution shift. In addition, classification-head variants substantially reduce inference time compared to generation-based inference. Overall, our results show that reasoning-oriented distillation combined with response stabilization makes compact open-source models more practical and reliable for X-CCD detection.

Related papers

• Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks [1.9800951131982487]
  We use Speculative Refinement (SpecRef) to warm-start a masked diffusion language model from an AR draft.<n>We evaluate SpecRef across six benchmarks (HumanEval, MBPP, GSM8K, BBH, ARC-Challenge, HellaSwag) with three distinct evaluation protocols.<n>These observations apply to any multi-stage or non-autoregressive generation pipeline and point toward more diagnostic evaluation practices.
  arXiv  Detail & Related papers  (2026-06-25T18:52:24Z)
• Encoder-Decoder Manifold Alignment for Idempotent Generation [51.959208893243776]
  We argue that a key reason for this failure is a geometric mismatch between the manifold learned by the encoder and decoder.<n>We propose a new training framework that explicitly closes this gap by forcing the encoder and decoder to learn consistent representations of the same underlying data manifold.
  arXiv  Detail & Related papers  (2026-06-21T02:14:37Z)
• SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation [79.75755802397312]
  SpecSteer is an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning.<n>Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance.
  arXiv  Detail & Related papers  (2026-03-17T07:51:29Z)
• Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning [58.331709210563616]
  Thinking by Subtraction is a confidence-driven contrastive decoding approach.<n>A small subset of low-confidence tokens disproportionately contributes to reasoning errors and unnecessary output expansion.<n>Our method, Confidence-Driven Contrastive Decoding, detects low-confidence tokens during decoding and intervenes at these positions.
  arXiv  Detail & Related papers  (2026-02-20T14:13:22Z)
• Reliability Under Randomness: An Empirical Analysis of Sparse and Dense Language Models Across Decoding Temperatures [0.0]
  We investigate whether conditional computation in sparse MoE models amplifies decoding-induced randomness, leading to reduced reliability as temperature increases.<n>Results demonstrate that the sparse instruction-tuned model exhibits stability comparable to the dense instruction-tuned model across all decoding temperatures.<n>We discuss the implications of these results for deploying sparse language models in reliability-critical applications.
  arXiv  Detail & Related papers  (2026-01-02T18:10:10Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• RoParQ: Paraphrase-Aware Alignment of Large Language Models Towards Robustness to Paraphrased Questions [0.0]
  Large Language Models (LLMs) often exhibit inconsistent behavior when answering paraphrased questions.<n>We introduce RoParQ, a benchmark to evaluate cross-paraphrase consistency in closed-book multiple-choice QA.<n>We also propose XParaCon, a novel evaluation metric that quantifies a model's robustness.
  arXiv  Detail & Related papers  (2025-11-26T16:40:53Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Extractive Fact Decomposition for Interpretable Natural Language Inference in one Forward Pass [4.990228412613982]
  JEDI is an encoder-only architecture that jointly performs atomic fact decomposition and interpretable inference.<n>To facilitate training, we produce a large corpus of synthetic rationales covering multiple NLI benchmarks.<n>Our findings show that interpretability and robust generalization in NLI can be realized using encoder-only architectures and synthetic rationales.
  arXiv  Detail & Related papers  (2025-09-23T11:30:42Z)
• TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generation [31.231916859341865]
  TrustRAG is a framework that systematically filters malicious and irrelevant content before it is retrieved for generation.<n>TrustRAG delivers substantial improvements in retrieval accuracy, efficiency, and attack resistance.
  arXiv  Detail & Related papers  (2025-01-01T15:57:34Z)
• AdaCCD: Adaptive Semantic Contrasts Discovery Based Cross Lingual Adaptation for Code Clone Detection [69.79627042058048]
  AdaCCD is a novel cross-lingual adaptation method that can detect cloned codes in a new language without annotations in that language. We evaluate the cross-lingual adaptation results of AdaCCD by constructing a multilingual code clone detection benchmark consisting of 5 programming languages.
  arXiv  Detail & Related papers  (2023-11-13T12:20:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.