LoRA-MME: Multi-Model Ensemble of LoRA-Tuned Encoders for Code Comment Classification

• URL: http://arxiv.org/abs/2603.03959v2
• Date: Thu, 05 Mar 2026 18:19:21 GMT
• Title: LoRA-MME: Multi-Model Ensemble of LoRA-Tuned Encoders for Code Comment Classification
• Authors: Md Akib Haider, Ahsan Bulbul, Nafis Fuad Shahid, Aimaan Ahmed, Mohammad Ishrak Abedin,
• Abstract summary: We present LoRA-MME, a Multi-Model Ensemble architecture for multi-label classification.<n>Our approach addresses the multi-label classification challenge across Java, Python, and Pharo.<n>Our tool achieved an F1 Weighted score of 0.7906 and a Macro F1 of 0.6867 on the test set.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Code comment classification is a critical task for automated software documentation and analysis. In the context of the NLBSE'26 Tool Competition, we present LoRA-MME, a Multi-Model Ensemble architecture utilizing Parameter-Efficient Fine-Tuning (PEFT). Our approach addresses the multi-label classification challenge across Java, Python, and Pharo by combining the strengths of four distinct transformer encoders: UniXcoder, CodeBERT, GraphCodeBERT, and CodeBERTa. By independently fine-tuning these models using Low-Rank Adaptation(LoRA) and aggregating their predictions via a learned weighted ensemble strategy, we maximize classification performance without the memory overhead of full model fine-tuning. Our tool achieved an F1 Weighted score of 0.7906 and a Macro F1 of 0.6867 on the test set. However, the computational cost of the ensemble resulted in a final submission score of 41.20%, highlighting the trade-off between semantic accuracy and inference efficiency.

Related papers

• Cost-Aware Model Selection for Text Classification: Multi-Objective Trade-offs Between Fine-Tuned Encoders and LLM Prompting in Production [0.0]
  Large language models (LLMs) have demonstrated strong capabilities in open-ended reasoning and generative language tasks.<n>For structured text classification problems with fixed label spaces, model selection is often driven by predictive performance alone.<n>We show that fine-tuned encoder-based models from the BERT family achieve competitive, and often superior, classification performance.
  arXiv  Detail & Related papers  (2026-02-06T03:54:28Z)
• Confidence-Credibility Aware Weighted Ensembles of Small LLMs Outperform Large LLMs in Emotion Detection [0.2864713389096699]
  This paper introduces a confidence-weighted, credibility-aware ensemble framework for text-based emotion detection.<n>Our approach combines architecturally diverse small transformer-based large language models (sLLMs) - BERT, RoBERTa, DistilBERT, DeBERTa, and ELECTRA.<n>Experiments on the DAIR-AI dataset demonstrate that our credibility-confidence ensemble achieves a macro F1 score of 93.5 percent.
  arXiv  Detail & Related papers  (2025-12-19T14:33:14Z)
• Chart2Code-MoLA: Efficient Multi-Modal Code Generation via Adaptive Expert Routing [20.521717930460692]
  C2C-MoLA is a framework that synergizes Mixture of Experts (MoE) with Low-Rank Adaptation (LoRA)<n>LoRA enables parameter-efficient updates for resource-conscious tuning.<n>Experiments on Chart2Code-160k show that the proposed model improves generation accuracy by up to 17%.
  arXiv  Detail & Related papers  (2025-11-28T16:23:04Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Datarus-R1: An Adaptive Multi-Step Reasoning LLM for Automated Data Analysis [0.0]
  We present Datarus-R1-14B, a language model fine-tuned from Qwen 2.5-14B-Instruct to act as a virtual data analyst and graduate-level problem solver.<n>Datarus is trained not on isolated question-answer pairs but on full analytical trajectories including reasoning steps, code execution, error traces, self-corrections, and final conclusions.
  arXiv  Detail & Related papers  (2025-08-18T21:58:18Z)
• Learning to Select MCP Algorithms: From Traditional ML to Dual-Channel GAT-MLP [8.6217237605907]
  No single maximum clique algorithm consistently performs best across all instances.<n>We propose a learning-based framework that integrates both traditional machine learning and graph neural networks.
  arXiv  Detail & Related papers  (2025-08-11T14:09:58Z)
• University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection [1.2564343689544841]
  This paper focuses on multilabel emotion classification across 28 languages.<n>We explore two main strategies: fully fine-tuning transformer models and classifier-only training.<n>Our findings suggest that training a classifier on top of prompt-based encoders such as mE5 and BGE yields significantly better results than fully fine-tuning XLMR and mBERT.
  arXiv  Detail & Related papers  (2025-05-22T09:42:11Z)
• Collab: Controlled Decoding using Mixture of Agents for LLM Alignment [90.6117569025754]
  Reinforcement learning from human feedback has emerged as an effective technique to align Large Language models.<n>Controlled Decoding provides a mechanism for aligning a model at inference time without retraining.<n>We propose a mixture of agent-based decoding strategies leveraging the existing off-the-shelf aligned LLM policies.
  arXiv  Detail & Related papers  (2025-03-27T17:34:25Z)
• MatFormer: Nested Transformer for Elastic Inference [91.45687988953435]
  MatFormer is a novel Transformer architecture designed to provide elastic inference across diverse deployment constraints.<n>MatFormer achieves this by incorporating a nested Feed Forward Network (FFN) block structure within a standard Transformer model.<n>We show that a 850M decoder-only MatFormer language model (MatLM) allows us to extract multiple smaller models spanning from 582M to 850M parameters.
  arXiv  Detail & Related papers  (2023-10-11T17:57:14Z)
• Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler [103.97487121678276]
  Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs. We propose a novel Knowledge-driven Differential Filter Sampler(KDFS) with Masked Filter Modeling(MFM) framework for filter pruning.
  arXiv  Detail & Related papers  (2023-07-01T02:28:41Z)
• Mixture Model Auto-Encoders: Deep Clustering through Dictionary Learning [72.9458277424712]
  Mixture Model Auto-Encoders (MixMate) is a novel architecture that clusters data by performing inference on a generative model. We show that MixMate achieves competitive performance compared to state-of-the-art deep clustering algorithms.
  arXiv  Detail & Related papers  (2021-10-10T02:30:31Z)
• Pack Together: Entity and Relation Extraction with Levitated Marker [61.232174424421025]
  We propose a novel span representation approach, named Packed Levitated Markers, to consider the dependencies between the spans (pairs) by strategically packing the markers in the encoder. Our experiments show that our model with packed levitated markers outperforms the sequence labeling model by 0.4%-1.9% F1 on three flat NER tasks, and beats the token concat model on six NER benchmarks.
  arXiv  Detail & Related papers  (2021-09-13T15:38:13Z)
• Gestalt: a Stacking Ensemble for SQuAD2.0 [0.0]
  We propose a deep-learning system that finds, or indicates the lack of, a correct answer to a question in a context paragraph. Our goal is to learn an ensemble of heterogeneous SQuAD2.0 models that outperforms the best model in the ensemble per se.
  arXiv  Detail & Related papers  (2020-04-02T08:09:22Z)
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction [75.16836697734995]
  We propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS) AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service.
  arXiv  Detail & Related papers  (2020-03-25T06:53:54Z)
• Learning Autoencoders with Relational Regularization [89.53065887608088]
  A new framework is proposed for learning autoencoders of data distributions. We minimize the discrepancy between the model and target distributions, with a emphrelational regularization We implement the framework with two scalable algorithms, making it applicable for both probabilistic and deterministic autoencoders.
  arXiv  Detail & Related papers  (2020-02-07T17:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.